###### Abstract  
Dimensionality reduction plays an important role in interpreting and visualizing high-dimensional data. Previous methods for data visualization overestimate the local structure and lack the consideration of global preservation. In this study, we develop a Gaussian process latent variable model (GP-LVM) for data visualization. GP-LVMs are one of the frameworks of principal component analysis and preserve the global structure effectively. The drawbacks of GP-LVMs are the absence of local structure preservation and the use of low-expressive kernel functions. Therefore, we introduce regularization for local preservation and an expressive kernel function into GP-LVMs to overcome these limitations. As a result, we reflect the global and local structures in low-dimensional representations, improving the reliability and visibility of embeddings. We conduct qualitative and quantitative experiments comparing baselines and state-of-the-art methods on image and text datasets. data visualization; dimensionality reduction; Gaussian process; neighborhood graph +
Footnote †: journal:  
are suitable for data preprocessing and visualization and have gained significant attention for dimensionality reduction as a counterpart to graph-based approaches [19; 20; 21; 22; 23]. The kernel function and prior distribution used in GP-LVMs influence the quality of low-dimensional representations. The kernel function determines the properties of the embedding space, and the prior distribution corresponds to the regularization term of low-dimensional embedding. Previously, researchers have proposed prior distributions for various purposes, such as time-series analyses [24; 25], WiFi localization [26; 27], and face verification [28]. Furthermore, many kernel functions have been developed in the regression literature [29; 30; 31] and have achieved many successful results. However, in GP-LVMs, the prior distribution is used to express supervised information [28; 32; 33] and is not designed for unsupervised cases typical in data visualization. Furthermore, the kernel functions used in GP-LVMs are limited to the class of homogeneous kernels, such as the squared exponential (SE) kernel and Matern families, which have limited expressions. Therefore, we require a comprehensive approach to GP-LVMs for data visualization purposes. In this study, we develop a GP-LVM for data visualization. Our main contribution is the development of a stationary kernel for GP-LVMs and a prior distribution for unsupervised cases. The detail of each component is as follows. **Kernel function. ** We focus on the spectral mixture (SM) kernel [30] and introduce it into a GP-LVM. Generally, the SE kernel and Matern kernel are applied to GP-LVMs, which are homogeneous kernels that only depend on the distance between latent features. In contrast, the SM kernel is a broader class kernel that depends on not only the distance but the periodicity of input features. Although the SM kernel has strong expressiveness compared to the SE and Matern kernels, it has been applied to regression models and not to GP-LVMs. In this study, we develop the SM kernel for GP-LVMs and reveal its high ability for pattern discovery and visualization. **Prior distribution. ** We focus on the use of a locally estimated manifold in the prior distribution. Local structures, such as the cluster structure, are crucial to improving the visibility of a low-dimensional representation. Therefore, we regularize low-dimensional embedding with the local estimation of the data manifold. Specifically, we locally approximate it with a neighborhood graph and regularize the low-dimensional representation with it. Through this strategy, we consider local structures and improve the visibility of low-dimensional representation. The regularization based on the graph Laplacian in supervised cases has been previously studied as the Gaussian process latent random field (GPLRF) [33], and we apply the GPLRF scheme in an unsupervised manner. The remainder is as follows.