# Compression of Vehicle Trajectories with a Variational Autoencoder
## 4 Methodology
### Convolutional Variational Autoencoder  
Variational Autoencoder inherently has an explicit regularization during the training process [35]. VAEs encode the input as a distribution over the latent space instead of encoding it as a point. The context vector is then sampled from this distribution, the sampled context is decoded, and the reconstruction error is computed. One data point from the data set is denoted by \(x\) and the encoded representation, called \"context vector\" is \(z\). Let's denote the prior distribution over the latent space by \(p(z)\). Probabilistic decoder: This is defined by \(p(x|z)\), and it describes the distribution of the decoded variable given the encoded variable. Probabilistic encoder: This is defined by \(p(z|x)\), and it describes the distribution of the encoded variable given the decoded variable. The probabilistic encoder and decoder concept is not enough to solve the regularization problem regarding the content generation. It is easy to see that the model is not prevented from returning distributions like Dirac delta or punctual functions behave like classic models leading to overfitting. In the next paragraphs, the details of the mathematical concepts of the VAE are covered, and the deduction of the correct loss function for the optimization is presented. Below, two assumptions are made about the distributions:  
\[p(z) \equiv\mathcal{N}(0,\mathbb{I}) \tag{3}\] \[p(x|z) \equiv\mathcal{N}(f(z),c\mathbb{I})\hskip 14.226378 ptf\in F;c>0\,. \tag{4}\]  
\(F\) denotes a set of functions that can be parametrized by a finite number of parameters, so it does not cover all the functions in general. The \(F\) elements can be approximated by neural networks, which is the key to finding the optimal decoding. On the other hand, the Bayesian theorem can unfold the probabilistic encoder  
\[p(z|x)=\frac{p(x|z)p(z)}{p(x)}=\frac{p(x|z)p(z)}{\int p(x|u)p(u)du} \tag{5}\]  
which is used in Equation (7) for the deduction of a practical formula. Otherwise, the integral in Equation (5) cannot be evaluated, so it cannot be used in practice. By means of the variational inference formulation, the \(p(z|x)\) encoder distribution is approximated by an other Gaussian \(q_{x}(z)\), where the mean and the variance are functions of variable \(x\):  
\[q_{x}(z)\equiv\mathcal{N}(g(x),h(x))\quad g\in G;h\in H\,. \tag{6}\]  
Here, \(G\) and \(H\) are sets of parametrized functions just like \(F\). Among these functions, one needs to find the best approximation of the \(g,h\) functions by determining their parameters. The best approximation is supposed to minimize the Kullback-Leibler divergence [36] between the approximation and the original \(p(z|x)\) distribution. The optimal \(g^{*}\) and \(h^{*}\) functions obtained by  
\[(g^{*},h^{*})=\operatorname*{arg\,min}_{(g,h)\in G\times H}KL(q_{x}(z),p(z|x) ). \tag{7}\]  
Here, Equation (5) is substituted and one gets  
\[(g^{*},h^{*})=\operatorname*{arg\,min}_{(g,h)\in G\times H}\left(\mathbb{E}( \log(q_{x}(z)))-\mathbb{E}\left(\log\frac{p(x|z)p(z)}{p(x)}\right)\right)\, \tag{8}\]  
\[(g^{*},h^{*})=\operatorname*{arg\,min}_{(g,h)\in G\times H}\left(\mathbb{E}( \log(q_{x}(z))-\mathbb{E}(\log p(z))-\mathbb{E}(\log p(x|z))+\mathbb{E}(\log p (x))\right). \tag{9}\]  
After rearranging the terms, the Kullback-Leibler divergence of \(q_{x}(z)\) and \(p(z)\) distributions appears, which can be calculated.