# Mixture Density Conditional Generative Adversarial Network Models (MD-CGAN)
## 3 The MD-CGAN Model Framework  
As with the GAN and CGAN methods, we consider a time series, \(y_{t}\). Our aim now is to infer the posterior distribution over some \(y_{t^{\prime}>t}\), conditioned on the set of observations, \(\mathbf{x}_{t}\). In order to form the posterior distribution, we model the full conditional density \(p(y_{t^{\prime}}|\mathbf{x}_{t})\) as an adversarial network. To achieve this, we use a Mixture Density Network (MDN) model [16] for the generator \(G\). The inputs to the generator network are as per the CGAN approach, \(\mathbf{x}_{t}\) and \(\mathbf{z}_{g}\), where \(\mathbf{z}_{g}\) is, as before, a collection of samples from a normal distribution, \(p(z_{n})_{g}=\mathcal{N}(0,\mathrm{var}_{\mathrm{data}})\). The outputs of \(G_{t^{\prime}}(\mathbf{x}_{t},\mathbf{z}_{g})\) are now, however, the parameters of the Gaussian mixture posterior over the forecast. This mixture has mixing  
Figure 1: Schematic of (**a**) GAN and (**b**) CGAN models, showing Generator and Discriminator components and associated variables. coefficient, standard deviation, and mean for the \(i\)-th component denoted as \(\alpha_{i}\), \(\sigma_{i}\), and \(\mu_{i}\), respectively. As first proposed in Reference [16], we achieve this by using latent variables \(\mathbf{s}=\{\mathbf{s}_{\alpha},\mathbf{s}_{\sigma},\mathbf{s}_{\mu}\}\), conditioned on the inputs. The mapping from \([\mathbf{x}_{t},\mathbf{z}_{g}]\mapsto\mathbf{s}\mapsto\{\mathbf{a}_{i}, \sigma_{i},\mu_{i}\}\) is modeled via our network. As the mixings must satisfy \(\sum_{i}\alpha_{i}=1\), we map \(\mathbf{s}_{\alpha}\) to \(\mathbf{\alpha}\) via the _softmax_ function, where \(\mathbf{\alpha}_{i}=\frac{\mathbf{exp}(\mathbf{s}_{\alpha i})}{\frac{\mathbf{ \gamma}}{\mathbf{\sigma}}\mathbf{exp}(\mathbf{s}_{\alpha i},\mathbf{\prime})}\). The elements of \(\sigma\) are strictly positive, so we adopt \(\sigma_{i}=\mathbf{exp}(\mathbf{s}_{\sigma,i})\). Finally, the means can be mapped directly from the latent variables; hence, \(\mathbf{\mu}_{i}=\mathbf{s}_{\mu,i}\). Schematically, the MD-CGAN method is depicted in Figure 2.  
The above formalism allows us to directly model the predictive likelihood conditioned on an input, and the likelihood of \(G\), conditioned on the observations \(\mathbf{x}_{t}\) and samples \(\mathbf{z}_{g}\) as:  
\[\mathcal{L}(G_{t^{\prime}}(\mathbf{x}_{t},\mathbf{z}_{g}))=\sum_{i=1}^{m} \alpha_{i}(\mathbf{x}_{t},\mathbf{z}_{g})\mathcal{N}_{i}(y_{t^{\prime}}|\mu_{ i}(\mathbf{x}_{t},\mathbf{z}_{g}),\sigma_{i}(\mathbf{x}_{t},\mathbf{z}_{g})), \tag{1}\]  
where \(m\) is the number of mixture components. As in the CGAN model, the discriminator, \(D\), is also conditioned on \(\mathbf{x}_{t}\). The input to the discriminator model is, by design, \(\mathbf{x}_{t}\sqrt{2\pi}\sigma_{a}\mathcal{L}(y_{t^{\prime}})\), where \(\sigma_{a}\) is the standard deviation of the set of observed \(y_{t}\) (We note that the GAN approach is not sensitive, within reason, to this value (it is, in effect, a constant in the update equations), and we discuss its setting later in the paper.). For true values of \(y_{t^{\prime}}\), \(\sqrt{2\pi}\sigma_{a}\mathcal{L}(y_{t^{\prime}})\) is maximized. The generator tries to 'fool' the discriminator by generating \(G_{t^{\prime}}\) such that the \(\sqrt{2\pi}\sigma_{a}\mathcal{L}(G_{t^{\prime}})\) is maximized. The loss function for the generator, \(L_{G}\), is as in Equation (2). The discriminator network, on the other hand, attempts to differentiate between true \(y_{t^{\prime}}\) values and the pseudo-values created by the generator. The loss function for the discriminator, \(L_{D}\), is as in Equation (3). We note that the lowest value of the discriminator loss is achieved when \(\sqrt{2\pi}\sigma_{a}\mathcal{L}(y_{t^{\prime}})\) is maximal (unity), and \(\mathcal{L}(G_{t^{\prime}}(\mathbf{x}_{t},\mathbf{z}_{g}))\) is minimal (zero).