# Simulating Intraday Electricity Consumption with ForGAN
## 2 GANs and ForGAN: Idea, Structure and Algorithms
### What Is a GAN?--The Mathematical Framework  
As can be indicated from the heuristic example, a GAN has two ingredients or players. The first one (_the generator_) seeks to learn a probability distribution from a data sample. The generator will then be able to create (multivariate) random numbers with this distribution from (multivariate) standard Gaussian-distributed random numbers. For this task, a differentiable function \(G(z):=G(z;\theta_{G})\), which is represented by a deep neural network (multiple layer perceptron) with a parameter set \(\theta_{G}\), will be fitted to the empirical distribution of the data sample.  
The second one (_the discriminator_) seeks to find a discrimination strategy between data coming from an unknown distribution, the training data, and data that are generated by the generator. Data points will be classified as _real_ or as _fake_, depending which of the two alternatives will be assigned a higher probability by the discriminator. The discriminator is given by a neural network \(D(x):=D(x,\theta_{D})\) with a parameter set \(\theta_{D}\). \(D(x)\) should output the probability that \(x\) is from the original dataset, which we denote by \(x\sim p_{data}\). It can be seen as a generalization of a linear discriminant analysis.  
While D will be trained to maximize the log-transformed probability \(\log D(x)\) that \(x\) is from the data of the training set, G is trained to minimize the (log-transformed) rejection probability \(\log(1-D(G(z)))\) for \(z\) being from the noise distribution, which is denoted by \(z\sim p_{z}\), where \(p_{z}\) is a (multi-dimensional) Gaussian distribution with zero mean and unit variance.  
The joint effort by the generator and the discriminator is modeled through a two-player min-max game between them of the following form (see [14]):  
\[\min_{\theta_{G}}\max_{\theta_{D}}V(G,D)=\min_{\theta_{G}}\max_{\theta_{D}} \mathbb{E}_{x\sim p_{data}}[\log D(x)]+\mathbb{E}_{z\sim p_{z}}[\log(1-D(G(z) ))], \tag{1}\]  
Figure 1: Swiss chocolate GAN analogy: The discriminator (chocolate expert) and generator (chocolate company) cooperate to create Swiss-style chocolate [13].  
where \(V(D,G)\) is a binary cross-entropy function, commonly used in binary classification problems [15]. It is worth noting that the loss function in the min-max problem (1) has the aim to match the generator's distribution \(p_{g}\) (which is the result of transforming \(p_{z}\) with \(G(.)\)) with the distribution of the real data \(p_{data}\). The dependence of D and G from \(\theta_{G},\theta_{D}\) on the right side of the min-max problem is omitted for the simplicity of notation.  
Of course, the expectations in (1) have to be estimated by the corresponding empirical means. The estimation is always based on subsamples (_mini batches_) of real and generated data of the same size \(m\). Figure 2 below illustrates the basic architecture of a GAN. Here, the attribute _generated_ is replaced by _fake_, as is often done in the literature. A further comment on the relation between the different objective functions of the discriminator and the generator is provided in Section 2.3 below. For more details on the mathematical background of GANs, we refer to [16].