## 3 Proposed VAE Using Context Information  
Although the compressed vector \(\mathbf{z}\) can be used as a useful global descriptor, it is insufficient to cope with environmental changes. To find the same place obtained from different environments, external factors such as weather or seasonal changes should be removed from the vector \(\mathbf{z}\). If the vector \(\mathbf{z}\) is divided into the condition-invariant feature \(\mathbf{z}_{p}\) and the condition-sensitive feature \(\mathbf{z}_{c}\), we would be able to reliably distinguish places even in changing environments using only the condition-invariant feature \(\mathbf{z}_{p}\). To achieve this goal, we assume that observed images are affected by both structural information \(\mathbf{p}\) such as unique landmarks, and context information \(\mathbf{c}\) due to environmental changes such as light or weather changes. Since structural information is robust and context information is sensitive to environmental changes, each information is contained in the condition-invariant feature \(\mathbf{z}_{p}\) and the condition-sensitive feature \(\mathbf{z}_{c}\), respectively. To divide the latent feature \(\mathbf{z}\) into \(\mathbf{z}_{p}\) and \(\mathbf{z}_{c}\), we propose a structure for generating the context vector \(c\) from \(\mathbf{z}_{c}\) and the image from both \(\mathbf{z}_{p}\) and \(\mathbf{z}_{c}\). Therefore, the generative model is changed from \(p_{\theta}(\mathbf{x}|\mathbf{z})\) to \(p_{\theta,\varphi}(\mathbf{x},\mathbf{c}|\mathbf{z}_{p},\mathbf{z}_{c})\), and is factorized as the following:  
\[p_{\theta,\varphi}(\mathbf{x},\mathbf{c}|\mathbf{z}_{p},\mathbf{z}_{c})=p_{ \theta}(\mathbf{x}|\mathbf{z}_{p},\mathbf{z}_{c})\cdot p_{\varphi}(\mathbf{c }|\mathbf{z}_{c}) \tag{2}\]  
where \(\theta\) and \(\varphi\) are parameters of the generative model to generate \(\mathbf{x}\) and \(\mathbf{c}\), respectively. The comparison between the existing and proposed generative model is shown in Figure 3.  
Figure 2: The structure of the vanilla VAE composed of the encoder and the decoder. Then, the variational lower bound is also modified from \(\mathcal{L}(\theta,\phi;\mathbf{x})\) to \(\mathcal{L}(\theta,\phi,\varphi;\mathbf{x},\mathbf{c})\) on the marginal likelihood as follows:  
\[ \begin{split}\mathcal{L}(\theta,\phi,\varphi;\mathbf{x},\mathbf{c} )&=\mathbb{E}_{q_{\phi}(\mathbf{z}|\mathbf{x})}[\log p_{\theta, \varphi}(\mathbf{x},\mathbf{c}|\mathbf{z})]-D_{KL}(q_{\phi}(\mathbf{z}| \mathbf{x})||p_{\theta,\varphi}(\mathbf{z}))\\ &=\mathbb{E}_{q_{\phi}(\mathbf{z},\mathbf{z}_{c}|\mathbf{x})}[ \log p_{\theta}(\mathbf{x}|\mathbf{z}_{p},\mathbf{z}_{c})+\log p_{\varphi}( \mathbf{c}|\mathbf{z}_{c})]\\ &\qquad\qquad-D_{KL}(q_{\phi}(\mathbf{z}_{p},\mathbf{z}_{c}| \mathbf{x})||p_{\theta}(\mathbf{z}_{p},\mathbf{z}_{c}))p_{\varphi}(\mathbf{z }_{c}))\end{split}  \tag{3}\]  
In order to learn the probability distributions, our proposed structure named _C-VAE_ is shown in Figure 4. The encoding part is considered as the inference model \(q_{\phi}(\mathbf{z}_{p},\mathbf{z}_{c}|\mathbf{x})\), and the decoding part is the generative model \(p_{\theta}(\mathbf{x}|\mathbf{z}_{p},\mathbf{z}_{c})\) and \(p_{\varphi}(\mathbf{c}|\mathbf{z}_{c})\). A detailed examination of this structure reveals the following characteristics in comparison with the existing VAE. The reconstruction of the input image \(\mathbf{x}\) is the same as the existing structure. The difference is that \(\mathbf{z}_{c}\), a subset of \(\mathbf{z}\), is used not only to reconstruct \(\mathbf{x}\), but also to create the context vector \(\mathbf{c}\). During the learning process, information that  
Figure 4: The structure of the C-VAE for feature extraction in changing environments. Figure 3: The comparison between (**a**) existing model and (**b**) proposed graphical models for data generation. Solid lines denote the generative model and dashed lines denote the recognition model.