# Generative Modeling for Imbalanced Credit Card Fraud Transaction Detection
## 3 Generative Models  
Generative models are a type of deep learning architecture used to capture the underlying structure of data and generate synthetic data by simulating the distribution of the real data. Initially popularized for image generation due to their remarkable results, these models have garnered significant interest from researchers exploring new applications, such as dimensionality reduction and feature selection [32]. In this paper, we leverage the capabilities of generative models to address the issue of data imbalance in our dataset. Generally, this approach works as follows: given a training set \(X_{\text{train}}\) and a set of parameters \(\theta\), a model can be constructed to estimate the probability distribution of the data. The likelihood is the probability that the model assigns to the training data for a dataset containing \(m\) samples of \(x^{(i)}\):  
\[\prod_{i=1}^{m}p_{\text{model}}(x^{(i)};\theta) \tag{1}\]  
The maximum likelihood method provides a way to compute the parameters \(\theta\) that can maximize the likelihood of the training data [33]. To simplify the optimization, we take the logarithm of the likelihood in Equation (1) to express the probabilities as a sum rather than a product:  
\[\theta^{*}=\arg\max_{\theta}\sum_{i=1}^{m}\log p_{\text{model}}(x^{(i)};\theta) \tag{2}\]  
If the true data distribution \(p_{\text{data}}\) lies within the family of distributions represented by \(p_{\text{model}}(x;\theta)\), the model can accurately approximate \(p_{\text{data}}\). However, in practice, the true distribution is not accessible, and only the training data are available for modeling [34]. Thus, the models must define their density function and find the \(p_{\text{model}}(x;\theta)\) that maximizes the likelihood. Generative models produce synthetic data by learning the probability distribution of the observed data and generating new samples from this learned distribution. The process typically involves two key components:  
1. **Latent Variables:** These are unobserved variables that capture the underlying factors of variation in the data. Let \(\mathbf{z}\) represent a vector of latent variables, which are typically sampled from a simple prior distribution \(p(\mathbf{z})\). This prior is often chosen to be a standard normal distribution, i.e., \(\mathbf{z}\sim\mathcal{N}(\mathbf{0},\mathbf{I})\).
2. **Generative Function:** This function, parameterized by \(\theta\), maps the latent variables \(\mathbf{z}\) to the data space. The generative process can be expressed as \(\mathbf{x}=G(\mathbf{z};\theta)\), where \(G\) is a neural network or another function that transforms the latent space into the data space, generating synthetic data samples \(\mathbf{x}\). The objective of training a generative model is to approximate the true data distribution \(p_{\text{data}}(\mathbf{x})\) by learning the model distribution \(p_{\text{model}}(\mathbf{x})\). This involves optimizing the model parameters \(\theta\) such that the synthetic data distribution \(p_{\text{model}}(\mathbf{x})\) closely matches the real data distribution. Meanwhile, generative models are powerful tools for data generation, but their application to imbalanced data problems comes with inherent challenges, particularly mode collapse and instability during training. Mode collapse occurs when the generator learns to produce a limited set of outputs, failing to capture the full diversity of the target distribution, which can undermine the quality of the synthetic data generated for the minority class. Additionally, the adversarial nature of those models can lead to training instability, where the generator and discriminator fail to converge, resulting in poor-quality synthetic samples. In the rest of this section, we describe the proposed models for handling the imbalance issue.