# Multi-Output Regression with Generative Adversarial Networks (MOR-GANs)
### Related Work  
Generative models were originally developed with the aim of creating a network that could generate realistic examples, that is examples that appear to be drawn from the distribution which was used to train the model. A powerful generative model is the Generative Adversarial Network (GAN) introduced in 2014 by [PERSON] et al. (2014). GANs have quickly become one of the most popular generative models and are widely used in image processing [PERSON] et al. (2014) where they are well known for generating images that are capable of tricking the human eye into believing that it is seeing genuine data [PERSON] et al. (2014). Instead of learning a mapping between an input and output determined by training data, these models attempt to learn the distribution underlying the training data (in fact, they learn a mapping from a simple distribution to the more complex distribution which describes the training data). This property is desirable as we would like to avoid extrapolating because it can lead to unreliable results. A GAN consists of two neural networks, a generator and a discriminator that are trained simultaneously according to a min-max game. The generator and discriminator adopt the structure of popular neural networks [PERSON] et al. (2014); [PERSON] et al. (2014); [PERSON] et al. (2014). Although many studies have explored the idea of using GANs when manipulating or identifying images, little research currently exists around implementing GANs to generate non-image data with targeted distributions. One exception is [PERSON] et al. (2018) who apply GANs to the time series prediction of fluid flow. Furthermore, GANs have shown to be able to perform well even with small samples of data [PERSON] et al. (2014), making them a reliable technique and suitable for regression in these circumstances. Since their introduction in 2014, a number of variants have been developed, including the Wasserstein GAN (WGAN) (WGAN) (WGAN, 2014; 2015). This particular flavour of GAN was introduced to address the problems of mode collapse and vanishing gradients [PERSON] et al. (2014) from which the GAN [PERSON] et al. (2014) and DCGAN [PERSON] et al. (2014) are known to suffer.  
GAN methods are not widely used for regression in the literature, with the exception of [PERSON] et al. (2015) and [PERSON] et al. (2016). [PERSON] et al. (2015) apply Conditional GAN (CGAN) to a number of datasets, including one which predicted property prices in California and another which predicted the control action on the ailerons given the status of the aeroplane. [PERSON] et al. (2016) apply a semi-supervised Cycle Wasserstein Regression GAN (CWR-GAN) to biomedical applications such as predicting a patient's response to treatment. Both articles showed good results, but both commented on the additional training time and training complexity exhibited by the GAN models in comparison with other methods. The CGAN and the CWR-GAN both have a different structure to the WGAN implemented here. Our WGAN (as with a standard GAN) generates a sample from random values (the input to the WGAN), whereas the CGAN and CWR-GAN have inputs and outputs of the same dimension, although the input can have additional variables corresponding to noise or constraints. Therefore CGANs and the CWR-GAN can be more straightforward to use for regression and time series modelling.  
We compare our GAN approach with regression performed by Gaussian Process Regression (GPR). GPR has become an effective, non-parametric Bayesian approach that can be applied to regression problems and can be utilised in exploration and exploitation scenarios [17]. Instead of inferring the distribution of parameters, non-parametric methods can directly predict the distribution of functions. Gaussian Process Regression starts with a set of prior functions based on a specified kernel. After incorporating some known function values (from the training dataset), a posterior distribution is obtained. The posterior can then be evaluated at points of interest (from the test dataset) [18].