## 4 The Model
### Proposed Model
This architectural approach offers a robust starting point, particularly effective in identifying local patterns and short-term temporal relationships in time data. Subsequently, the baseline model underwent modification through the replacement of the convolutional layers with GRUs, which are capable of capturing long-term temporal dependencies. The structure of this modified model is defined as follows:  
* Encoder: The encoder is constituted by a series of GRU layers, which process time sequences and generate a latent representation that encompasses both short- and long-term information. The dimensions of the GRU layers are specific and increase in value; the initial layers return the entirety of the sequence, while the final layer provides a single vector representing the entire sequence. Furthermore, a dropout is applied to regularise the model and prevent overfitting. Two dense layers are employed to calculate the mean and logarithmic variance of the latent vector. This is followed by a sampling layer that generates a sample from the latent space, utilising the same approach as that employed in the previous model. * Decoder: The decoder receives the latent vector as an input and utilises a RepeatVector to replicate it along the length of the sequence, thereby enabling the reconstruction of a sequence of a similar size to that of the original input. Subsequently, a series of GRU layers with inverse dimensions to those of the encoder is employed to decode the latent vector. Finally, a TimeDistributed dense layer is employed to reinstate the original feature size for each time step, enabling the reconstruction of the data sequence. This configuration introduces greater flexibility and learning capacity, enabling the model to more effectively adapt to the intricacies of financial time series. The optimisation of the model was sought through the exploration of several parameters, including the following:  
* The size of the latent vector, to achieve an appropriate balance between representational capacity and generalisation;
* The number and size of GRU layers;
* The learning rate;
* The dropout rate level, to prevent overfitting. Moreover, in this instance, the learning rate was configured using an exponential decay function, thereby enabling a dynamic adjustment of its value during the training process. In the final stage, the VAE architecture was further enhanced by the integration of a MCMC sampling process. The model retains the same structure as the VAE-GRU model, but implements a sampling process in latent space that differs from the common random extraction of a single sample. In this latter process, the following equation is used:  
\[z=z_{\text{mean}}+\epsilon\cdot\exp\Bigl{(}0.5\cdot z_{\text{log\_var}}\Bigr{)} \tag{6}\]  
where the \(z_{\text{mean}}\) is the mean of the latent space; the \(z_{\text{log\_var}}\) is the logarithmic variance of the latent space, and the term \(\epsilon\) represents random noise. This approach to single extraction may prove inadequate for representing the full range of variability and uncertainty present within the latent distribution. In this instance, a MCMC sampling approach is employed, whereby several samples are generated, thus enabling Monte Carlo estimation. In contrast to the previous method, each sample is now considered to be correlated with the others, through the introduction of a Markov dependency. This is achieved by utilising a Gated Recurrent Unit cell (GRUCell) to represent the transition dynamics. In this manner, each subsequent latent state is sampled following the preceding state, which is represented by a hidden state, \(h_{t}\). This approach enables the modelling of temporal correlation between successive samples across the latent space, which can be particularly convenient in the context of sequential data. In all models, the VAE loss function is defined by the combination of two components: the reconstruction loss and the KL loss.