# Multi-Output Regression with Generative Adversarial Networks (MOR-GANs)
## 2 Methods
### Regression with WGAN  
GANs are a type of generative network, the aim of which is to generate realistic looking samples that appear to have been drawn from the same distribution as the training data. The input to a WGAN (and GANs in general) is a set of random numbers (not related to the data) and the output contains the generated sample. Consider simple regression, where a relationship is sought between an independent variable \(x\) (also known as covariate or feature) and a dependent variable \(y\) (also known as a response). The WGAN is trained to produce both the independent and dependent variables of the regression problem \((x,y)\) as its output. In contrast, for feed-forward networks, the input of the network often takes the independent variable and the output, the dependent variable. Therefore, in order to specify a particular value for the independent variable, a prediction algorithm is introduced to the WGAN. Suppose we have trained the WGAN and wish to use it to predict the output at a given value of the independent variable \(x_{p}\). First, the latent variables (\(\mathbf{\alpha}\)) are set to random numbers. The generator is evaluated at these values, \(G(\mathbf{\alpha})\), producing a pair of values \((x,y)\)--the input and output or response in our regression problem. The difference between \(x\) and \(x_{p}\) is then minimised with respect to the latent variables. Once this is done, we assume that the output of the generator closely approximates \((x_{p},y_{p})\). The minimisation can be done efficiently by using the same software libraries that are used for back-propagation during training. This procedure means that we can generate multiple outputs for one input, \(x_{p}\), by starting from different random states for the latent variables, and we can therefore produce a distribution of values which reflect the uncertainty in the output \(y_{p}\). This procedure is detailed in Algorithm 1 and introduces a projection operator, Proj, which projects the output of the WGAN onto a space that contains only the variables that are to be constrained. For the example described in this paragraph, the projection operator would be represented by the matrix \([1\ 0]\) for an output of the generator in the form \((x,y)^{T}\). Simple regression (for a single independent variable), multiple regression (for more than one independent variable) and multi-output or multi-variate regression (for more than one dependent variable) can all be performed by the WGAN and demonstrated by the results in this paper. Due to the generative nature of the WGAN, both independent and dependent variables are contained in the output of the generator which means that, when randomly sampling the latent space of the generator to produce an output, we have no control over the particular value of independent variable. In order to specify particular values of independent variable(s), a prediction algorithm is used, described in Algorithm 1. So there are two ways of using the WGAN, either with random values for the independent variable or with constrained values:* **Random input:** Random variables are assigned to the latent space from which the generator of the WGAN yields a realistic output of a \(n\)-tuple of the independent and dependent variables associated with regression problem. By sampling the generator many times, this can be used to assess the probability density function learned by the generator. The value of the independent variable(s) cannot be controlled, however, as they are an output of the generator. Although random inputs allow us to see the distribution learned by the generator, having the facility to constrain the independent variables is an important feature. * **Constrained input:** An algorithm is used in conjunction with the (trained) WGAN to find predictions for given value(s) of the dependent variable(s). This results in a property similar to a GPR, where, for example, the independent variables are inputs of the GPR (and can be prescribed) and the outputs are dependent variables.