# Learning Kernel Stein Discrepancy for Training Energy-Based Models
## 5 Training Unnormalized Models with KSD
### PT KSD GAN (Kernel Stein Discrepancy Generative Adversarial Network with a Pulling-Away Term)  
In the above section, we analyzed the disadvantages of the autoencoder network architecture in EBGAN. Consequently, we optimized it to a standard neural network architecture. To ensure stability, we adjusted the hyperparameter \(m\) in Formula (17) to 1. The output of the discriminator is regarded as energy value of samples, which is the numerator of the explicit Gibbs density estimation. As stated in [25], the Kernel Stein Discrepancy (KSD) can measure the distance of different density distributions. That is, minimizing the \(KSD_{e}\) of the generator is equivalent to approaching fake samples, coinciding with the theoretical distribution of true samples. The modified formulation of \(L_{D}\) and \(L_{G}\) is presented as follows:  
\[ \begin{array}{l}L_{D}=D(x_{i})+(1-D(G(z)))\\ L_{G}=KSD_{e}(q,p)\end{array}  \tag{18}\]  
In the above formula, the generator and the discriminator are iteratively updated until, finally, the discriminator cannot tell whether the input sample is real or fake. We call this process of adversarial generation in Formula (18), KSD GAN. The discriminator is conducted as an energy-based model during training to discover the explicit Gibbs distribution of real samples. At the beginning, fake samples generated by the generator are very far from the density distribution region of real samples, and the discriminator can easily identify fake samples from real ones. During alternating iterations, samples generated by the generator are constantly narrowing \(KSD_{e}\) from the theoretical distribution of real samples, which is represented by the energy model distribution of the discriminator. As the number of iterations increases and the Kernel Stein Discrepancy shrinks, samples generated by the generator approximate the Gibbs distribution of true samples, fooling the discriminator into thinking they are real samples. In fact, this iterative process of KSD GAN can also be regarded as the confrontation process between the implicit generation model and the explicit generation model. The discriminator is trained as an explicit generation model, and its density distribution is the Gibbs distribution based on the energy model. Of course, we can use the sampler to sample from this Gibbs distribution, which we want to obtain. As an implicit generation model, the generator bypasses the estimation of the density distribution and generates imitation samples directly by reducing \(KSD_{e}\). GAN is notoriously difficult to train, and mode collapse often occurs. This is due to the widely dispersed density distribution of high-dimensional datasets, and there is a huge span between regions. It is very possible for the generator to find a local optimal and then stop optimizing. Then, samples generated by the generator are only one or several types, and the sample diversity is insufficient. Inspired by [36; 37], reducing the similarity and increasing orthogonalization of generated samples in energy-based models can avoid model collapse and speed up convergence. We add a repelling regularizer named the Pulling-away Term (_PT_) to the generator loss:  
\[PT(S)=\frac{1}{M(M-1)}\sum_{i}\sum_{j\
eq i}\left(\frac{S_{i}^{T}S_{j}}{\|S_{ i}\|\big{\|}S_{j}\big{\|}}\right)^{2} \tag{19}\]  
After adding the regular term to the generator, the loss \(L_{D}\) and \(L_{G}\) of KSD GAN are defined as follows:  
\[ \begin{array}{c}L_{D}=D(x_{i})+(1-D(G(z)))\\ L_{G}=\text{KSD}(q,p)+\mathbf{\ast}\ast PT\end{array}  \tag{20}\]  
In Formula (20), KSD GAN with the regular term _PT_ added is called PT KSD GAN. Our model structure of PT KSD GAN is shown in Figure 1.  
The figure above shows that our model is actually an enhanced GAN framework incorporating the Kernelized Stein Discrepancy. The data processing steps are described below:  
1. Two data streams, denoted as \(x_{i}\) and \(y_{j}\), are added to the system. The former is directed towards the discriminator, while the latter interfaces with the generator. 2.