## 3 Proposed Approach
### Problem Formulation  
We propose a model whose discriminator is viewed as an energy function that attributes low energies to the regions near the data manifold and higher energies to other regions, while its generator is seen as being trained to produce contrastive samples with minimal energies. With the term contrastive samples, we refer to a data point that causes an energy pull-up, such as an incorrect label in supervised learning and points from low data density regions in unsupervised learning. Our aim is to train the discriminator to assign high energies to these generated samples. Viewing the discriminator as an energy function, our proposed system allows us to use a wide variety of architectures and loss functionals--in addition to the usual binary classifier with logistic output--that have been introduced in the recently proposed \"Energy-Based Generative Adversarial Network\" model (EBGAN) [10]. However, our proposed model is a modified version of the EBGAN framework since we propose the use of an autoencoder architecture, with the energy being the reconstruction error, in place of the discriminator. Particularly, we suggest that rather than using a single bit of target information to train the model, the reconstruction-based output offers diverse targets for the discriminator. With the binary logistic loss, only two targets are possible, so, within a mini-batch, the gradients corresponding to different samples are most likely far from orthogonal. This leads to inefficient training, and reducing the mini-batch sizes is often not an option on current hardware. On the other hand, the reconstruction loss will likely produce very different gradient directions within the mini-batch, allowing for a larger mini-batch size without a loss of efficiency. Moreover, we decided to use the autoencoders since they have traditionally been used to represent energy-based models and arise naturally. When trained with some regularization terms, autoencoders have the ability to learn an energy manifold without supervision or negative examples. This means that even when an energy-based autoencoding model is trained to reconstruct a real sample, the discriminator contributes to discovering the data manifold by itself. To the contrary, without the presence of negative examples from the generator, a discriminator trained with binary logistic loss becomes pointless. Our proposed work is also inspired by the work of [PERSON] and [PERSON] [13]. However, it differs in the following way. Their approach uses a standard probabilistic GAN cast into an energy model (using Gibbs distributions), allowing them to learn a discriminator that models the distribution when Nash equilibrium is reached, but it still has mixing problems, especially with deep models that slow the learning. On the contrary, our approach gets rid of the probabilistic setting, while presenting the same Nash equilibrium as a standard GAN, but through a different and more generalized class of loss functionals. The computational procedure and structure of our proposed GAN model are depicted in Figure 1. We use differentiable functions \(D\) and \(G\) to represent the discriminator and the generator, respectively. Their inputs are real data \(x\) and random variables \(z\), respectively. \(G(z)\) represents the sample generated by \(G\) according to the distribution \(p_{data}\) of real data, and \(x^{\prime}\) corresponds to the novel synthetic data samples. If the input of discriminator \(D\) is from the real data \(x\), \(D\) should classify it to be true. If the input is from \(G(z)\), \(D\) should classify it to be false. The purpose of \(D\) is to achieve the correct classification of the data source, while the purpose of \(G\) is to approximate the performance of the generated data \(G(z)\) on \(D\) (i.e., \(D(G(z))\)) with the performance of real data \(x\) on \(D\) (i.e., \(D(x)\)). The adversarial optimization process improves the performance of \(D\) and \(G\) gradually.