## 3 Background for REDPACK
### Relativistic average GAN  
The applications of GAN, a groundbreaking framework for learning generative models, are varied. However, standard GAN (SGAN), originally proposed by [PERSON] et al. [10], is unstable in the learning phases, and optimizing the model is difficult. Many alternatives, including WGAN, have been proposed to mitigate this problem. GAN-based models can be broadly divided into integral probability metric (IPM)-based GANs and non-IPM-based GANs. Generally, IPM constraints provide stability for training the GAN. [PERSON] [13] analyzed the loss function of SGAN to analyze the limitations of the non-IPM-based GAN and proposed a relativistic GAN to address this. [PERSON] et al. [10] proved that GAN training attains the global optimum when the discriminator classifies real data with 0.5 probability. However, in many cases, the discriminator classifies both real and fake data as real, which is disadvantageous in training a good generator. This is because the discriminator is not aware that half of the data in the learning process is fake; the IPM-based GAN is relatively stable during learning because it implicitly accounts for this fact. From the perspective of divergence minimization, the discriminator is trained to increase \(D(x_{f})\), whereas \(D(x_{r})\) does not decrease accordingly, where \(x_{r}\) and \(x_{f}\) denote the real and fake data, respectively. To address this issue, [PERSON] [13] designed the output of the discriminator to depend on both the real and fake data:  
\[D(x_{r},x_{f})=\sigma(C(x_{r})-C(x_{f})) \tag{2}\]  
Where \(C(x)\) is the presumed critic (\(D(x)=\sigma(C(x))\)). Equation (2) can be interpreted as the discriminator's estimation of the probability of the given real data being more realistic than the fake data (\(D(x_{f},x_{r})\) can be interpreted in the opposite manner). If the discriminator is set according to Equation (2), unlike the generator of SGAN, which relies solely on the fake data, the generator from the relativistic GAN will depend on both the real and fake data. However, it has \(O(m^{2})\) complexity when calculating the loss (\(m\) means the size of the mini-batch) because it calculates the pair-wise differences of the critic between the real and fake data in the mini-batch. To solve this problem, [PERSON] [13] proposed a relativistic average GAN (RaGAN), which takes the expectation of the opposite type of data for some given data. The RaGAN uses the following loss functions to learn the discriminator and generator:  
\[\begin{split} L_{D}=&-\mathbb{E}_{x_{r}}[\log( \sigma(C(x_{r})-\mathbb{E}_{x_{f}}[C(x_{f})]))]-\mathbb{E}_{x_{f}}[\log(1- \sigma(C(x_{f})-\mathbb{E}_{x_{r}}[C(x_{r})]))]\\ L_{G}=&-\mathbb{E}_{x_{f}}[\log(\sigma(C(x_{f})- \mathbb{E}_{x_{r}}[C(x_{r})]))]-\mathbb{E}_{x_{r}}[\log(1-\sigma(C(x_{r})- \mathbb{E}_{x_{f}}[C(x_{f})]))]\end{split} \tag{3}\]  
where \(L_{D}\) and \(L_{G}\) represent the loss for learning the discriminator and generator, respectively. Equation (3) has \(O(m)\) complexity. The discriminator of RaGAN estimates the probability of some given data being more realistic than the opposite type of data, on average. Using different datasets, [PERSON] [13] showed that training RaGAN, compared to other GAN models, was faster and more reliable, and the generator of RaGAN generated high-quality fake data.