Title: Multi-View Oriented GPLVM: Expressiveness and Efficiency
Venue: NeurIPS 2025
Accepted: yes
Keywords: Multi-view representation learning

Kernel learning

Random Fourier feature
Abstract: The multi-view Gaussian process latent variable model (MV-GPLVM) aims to learn a unified representation from multi-view data but is hindered by challenges such as limited kernel expressiveness and low computational efficiency. To overcome these issues, we first introduce a new duality between the spectral density and the kernel function. By modeling the spectral density with a bivariate Gaussian mixture, we then derive a generic and expressive kernel termed Next-Gen Spectral Mixture (NG-SM) for MV-GPLVMs. To address the inherent computational inefficiency of the NG-SM kernel, we propose a random Fourier feature approximation. Combined with a tailored reparameterization trick, this approximation enables scalable variational inference for both the model and the unified latent representations. Numerical evaluations across a diverse range of multi-view datasets demonstrate that our proposed method consistently outperforms state-of-the-art models in learning meaningful latent representations.


Peer reviews:
comments: SUMMARY:
This paper proposes a novel NG-MVLVM model aimed at enhancing kernel expressiveness and scalable inference. The key contributions include a new duality between spectral density and kernel function, and a scalable random Fourier feature approximation combined with a reparameterization trick. Empirical validation showing superior performance over state-of-the-art multi-view learning methods.

STRENGTHS AND WEAKNESSES:
Strengths:
1. The proposed NG-SM kernel addresses limited kernel expressiveness, lack of flexibility, and high computational complexity of previous BSM kernels.
2. The duality between spectral density and kernel function provides a principled foundation for kernel design.
3. Experiments are conducted on diverse datasets to demonstrate the effectiveness of the proposed method.

Weaknesses:
1. It's hard to identify the contributions over previous work. The authors are suggested to summarize how each component differs from them.
2. Despite the huge amount of experiments, he paper does not fully analyze the individual contributions of the NG-SM kernel and the random Fourier feature approximation through ablation studies. The terms "kernel expressiveness", "time-varying correlations" are abstract and hard to quantify.  
3. The claimed efficiency is not empirically validated with runtime comparisons against other MVC methods.

QUESTIONS:
1. Is the bound of Theorem 4 tight and does it play an important role?
2. Are there guidences or requirements when choosing the kernel?

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 3
IMPACT: 2
ORIGINALITY: 2
DATE: 2025-07-03

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this work, the authors introduce a new multi-view Gaussian process variable model that incorporates a novel Next-Gen Spectral Mixture (NG-SM) kernel. This approach aims to address the limitations of kernel expressiveness and reduce the high computational complexity associated with the previous bivariate Spectral Mixture (BSM) kernel. The authors conduct experiments using synthetic data, (image, label) pairs, and wireless communication data. Their proposed method demonstrates state-of-the-art performance in terms of classification accuracy, training time, and data reconstruction.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n- The authors introduce a novel duality between the spectral density and the kernel function in the context of the multi-view Gaussian process latent variable model. The new kernel enhances both the expressiveness and computational efficiency compared to previous work. \n\n- The paper is well-written, accompanied by detailed supplementary material and shared code. The authors conduct experiments on a variety of datasets, reporting a range of results that include classification accuracy, training times, and data reconstruction capabilities. \n\n- The proposed approach achieves state-of-the-art performance when compared to prior Gaussian process latent variable models.\n\n**Weaknesses**\n- While the approach advocates for learning a model using multimodal data, the chosen experiments do not include sufficiently challenging multimodal datasets. In the (image, label) experiments, the use of multi-view techniques primarily serves as dataset augmentation since the new views are just rotations of the original images. Additionally, it is unclear to the reviewer whether the model is supposed to reconstruct the class labels for the images. Examples of more challenging multimodal datasets can be found in reference [1].\n\n- **Clarity** The reviewer believes that including diagrams illustrating the dependencies of the random variables, both overall and for each experimental setup, would enhance the understanding of your work. Although the authors provide experimental details in the supplementary materials, it remains unclear to the reviewer what Y represents for each dataset.\n\n- For the image-based datasets, a quantitative analysis of reconstruction quality is missing. How do the different approaches compare concerning data reconstruction?\n\n[1] Shi, Yuge, Brooks Paige, and Philip Torr. \"Variational mixture-of-experts autoencoders for multi-modal deep generative models.\" Advances in neural information processing systems 32 (2019).\n\nQUESTIONS:\n- Can the authors utilize neural network-based classifiers for evaluating classification accuracy results? \n- Why did the authors decrease the image size of the original datasets? How does the approach adjust to larger images? \n- How do the latent space sizes of the different models compare? Do they have similar dimensions? \n- For the wireless communication data, what do the ten different classes represent?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a novel and well-motivated multi-view representation learning method by addressing two key limitations of the existing MV-GPLVM model: the lack of kernel expressiveness and poor scalability.\n\nSTRENGTHS AND WEAKNESSES:\nThe authors propose a new Next-Gen Spectral Mixture (NG-SM) kernel, derived through a duality between the kernel and its spectral density modeled via bivariate Gaussian mixtures. To make the model scalable, they further design an efficient random Fourier feature approximation combined with a reparameterization strategy. The resulting model enables efficient variational inference and achieves strong performance across multiple heterogeneous multi-view datasets.\n\nQUESTIONS:\n1)\tWhile this paper claims that the proposed NG-SM kernel is capable of approximating any continuous kernel via Bochner duality and Gaussian mixture spectral density modeling, the theoretical guarantee is only loosely discussed. \n2)\tAlthough the method aims to learn informative shared latent variables across views, the paper does not offer enough interpretability or visualization analysis (e.g., t-SNE, clustering structure, or downstream task results). \n3)\tThe paper mentions that the complexity is reduced to O(NL²) using RFF-based kernel approximation, but it lacks empirical runtime comparisons with other methods.\n\nLIMITATIONS:\n1)\tWhile this paper claims that the proposed NG-SM kernel is capable of approximating any continuous kernel via Bochner duality and Gaussian mixture spectral density modeling, the theoretical guarantee is only loosely discussed. \n2)\tAlthough the method aims to learn informative shared latent variables across views, the paper does not offer enough interpretability or visualization analysis (e.g., t-SNE, clustering structure, or downstream task results). \n3)\tThe paper mentions that the complexity is reduced to O(NL²) using RFF-based kernel approximation, but it lacks empirical runtime comparisons with other methods.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-04"}