# Compression of Vehicle Trajectories with a Variational Autoencoder
## 4 Methodology
### Convolutional Variational Autoencoder
The last term does not depend on \(g\) and \(h\), so, generally, it gives only a constant contribution so that it can be omitted in the minimization problem:  
\[(g^{*},h^{*})=\operatorname*{arg\,min}_{(g,h)\in G\times H}\left(-\mathbb{E}( \log p(x|z)+KL(q_{x}(z),p(z))\right) \tag{10}\]  
\[(g^{*},h^{*})=\operatorname*{arg\,min}_{(g,h)\in G\times H}\left(\mathbb{E} \left(\frac{||x-f(z)||^{2}}{2c}\right)+KL(q_{x}(z),p(z))\right). \tag{11}\]  
The last equation expresses that the optimizer needs to find a balance between the reconstruction loss, which is the maximization of the log-likelihood for the \(p(x|z)\) and the KL divergence of \(q_{x}(z)\) and the prior which enforce the encoder to stay close to the standard normal distribution. This is a trade-off between how much one can rely on the data and how fair assumption is the prior. In case of the function \(f\), one should maximize the expected log-likelihood of output \(x\) given a \(z\) context vector sampled from \(g_{x}(z)\):  
\[f^{*}=\operatorname*{arg\,max}_{f\in F}\mathbb{E}(\log p(x|z))=\operatorname* {arg\,max}_{f\in F}\mathbb{E}\left(-\frac{||x-f(z)||^{2}}{2c}\right). \tag{12}\]  
Taking Equation (11) into account, the final optimization problem is formulated in maximizing the following expression with respect to the parameters of the three functions \((f,g,h)\):  
\[p(x|z) \equiv\mathcal{N}(f(z),c\mathbb{I})\hskip 14.226378 ptf\in F;c>0\,. \tag{4}\]0  
The mean square error approximates the first term. The second term is the Kullback-Leibler divergence between the normal distribution \(q_{x}(z)\) with a diagonal covariance matrix and the standard normal distribution. After performing the calculation, one gets the loss function for the VAE;  
\[p(x|z) \equiv\mathcal{N}(f(z),c\mathbb{I})\hskip 14.226378 ptf\in F;c>0\,. \tag{4}\]1In addition, using the mean squared error of the first derivatives gives a better and smoother result, thus the final loss function for the VAE with Equation (1) takes the following form:  
\[p(x|z) \equiv\mathcal{N}(f(z),c\mathbb{I})\hskip 14.226378 ptf\in F;c>0\,. \tag{4}\]2  
All the functions \(f,g,h\) are parametrized by the parameters of neural networks, and \(F,G,H\) sets are defined by their architectural design. Empirically determined parameters for the encoder and decoder are given in Tables 1 and 2.  
reconstruction loss and the regularization term. An average per sample is taken on the data set, so this value is the expected value of the reconstruction and regularization error of a sample trajectory with 2 \(\times\) 60-parameters. Furthermore, the learning curves of SAE, LSTM AE, and CVAE are presented in Figure 4 in the case of 10-dimensional latent space. A moving average smoothes the curves with a window size of 1000 epochs for the sake of better visibility. The Convolutional Sparse Autoencoder (SAE) described in Section 4.2 fails to converge during the training process. Using the regularization term in loss function Equation (2) gives a better result than without using it though the training loss freezes around 50-100 based on the learning rate and latent space dimension. In Figure 4, the red lines show that the SAE gets stuck in a local optimum during the optimization, unable to make that progress like the CVAE (cyan blue curves) does. Since the loss values are the sum of the losses along the 2 times 60 components of the meters' trajectory, the value above 35 means that the average deviation is around 0.5 m for each component. This is too big to say that SAE training is satisfactory. A similar finding can be made for the LSTM AE. The LSTM AE (Section 4.1) with the same regularization gives slightly better results regarding the loss values in \[p(x|z) \equiv\mathcal{N}(f(z),c\mathbb{I})\hskip 14.226378 ptf\in F;c>0\,. \tag{4}\]3 The LSTM can only approach CVAE's loss values with much greater complexity. Once the latent space dimension is chosen below 10, the training process can not converge, so the losses become unmanageable big values and extremely fluctuating.