Probabilistic Geometric Principal Component Analysis with application to neural data

Authors: Han-Lin Hsieh, Maryam Shanechi
Primary area: applications to neuroscience & cognitive science
Keywords: geometry, nonlinear manifold, factor analysis, dimensionality reduction, neural population activity

Abstract:
Dimensionality reduction is critical across various domains of science including neuroscience.  Probabilistic Principal Component Analysis (PPCA) is a prominent dimensionality reduction method that provides a probabilistic approach unlike the deterministic approach of PCA and serves as a connection between PCA and Factor Analysis (FA). Despite their power, PPCA and its extensions are mainly based on linear models and can only describe the data in a Euclidean coordinate system around the mean of data. However, in many neuroscience applications, data may be distributed around a nonlinear geometry (i.e., manifold) rather than lying in the Euclidean space around the mean. We develop Probabilistic Geometric Principal Component Analysis (PGPCA) for such datasets as a new dimensionality reduction algorithm that can explicitly incorporate knowledge about a given nonlinear manifold that is first fitted from these data. Further, we show how in addition to the Euclidean coordinate system, a geometric coordinate system can be derived for the manifold to capture the deviations of data from the manifold and noise. We also derive a data-driven EM algorithm for learning the PGPCA model parameters. As such, PGPCA generalizes PPCA to better describe data distributions by incorporating a nonlinear manifold geometry. In simulations and brain data analyses, we show that PGPCA can effectively model the data distribution around various given manifolds and outperforms PPCA for such data. Moreover, PGPCA provides the capability to test whether the new geometric coordinate system better describes the data than the Euclidean one. Finally, PGPCA can perform dimensionality reduction and learn the data distribution both around and on the manifold. These capabilities make PGPCA valuable for enhancing the efficacy of dimensionality reduction for analysis of high-dimensional data that exhibit noise and are distributed around a nonlinear manifold, especially for neural data.

Decision: accept (spotlight)
Reviewer scores: 8, 6, 8

Reviews and discussion:
<thread>
creation date:
1729871412807

thread_type:
official_review

<thread_object>
creator: reviewer_dfnu

<content>
summary:
This paper proposes a generalization of probabilistic principal component analysis (PPCA) to data that lies on a (nonlinear) manifold rather than in Euclidean space around the mean. 

The key contributions of the paper are (i) the introduction of the PGPCA model, (ii) an expectation-maximization algorithm to fit the proposed model and (iii) empirical experiments with real and simulated data to compare PGPCA with vanilla PPCA.

strengths:
- The problem of dimensionality reduction for data on a manifold is well motivated, and the proposed method closes the gap between PPCA and data on manifolds. 
- The proposed extension PPCA is innovative and well explained.
- The mathematical derivation of the EM-algorithm is concise and understandable, and the empirical experiments show promising results.

weaknesses:
- The simulated data seems to be favorable for PGPCA compared to PPCA as it lies on non-linear manifolds. It remains open how well PGPCA compares to PPCA for data in other settings, e.g. in linear subspaces.
- PGPCA is only compared to PPCA and not other - potentially stronger - baseline methods. Especially, comparisons with other dimension reduction methods for data on manifolds is missing.
- The evaluation on real data is rather limited. Only brain signals of two mice were used.

questions:
- Please add literature on dimensionality reduction for data on a manifold.
- How well does PGPCA work with other choices of $K(z)$ than geometric and Euclidean?
- What theoretical guarantees for the hypothesis test on EuCOV vs. GeCOV are there? (e.g. in terms of statistical power/or asymptotic level)
- How does PGPCA compare to PPCA for data in linear subspaces?
- How does PGPCA compare against stronger competitors?
- How well does PGPCA work on other real data?
- Can the i.i.d. assumptions in model (1) be relaxed? E.g. by allowing (temporal) dependence or slightly varying distributions?
- Can the assumption of normality be relaxed to allow for non-normal errors?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730656725168

thread_type:
official_review

<thread_object>
creator: reviewer_cyyb

<content>
summary:
The paper's main idea is the PPCA extension containing nonlinear manifold terms. The reason for this assumption is not clear. 
But the formulation and mathematical analysis are true.
Also, its performance should be seen in real data. However the experiments on real data are not clear and also they should apply it for more general data like well-known image data sets, or at least mention the class of real data that this method can work well for them in comparison with PPCA. By this version of experiments the importance and quality of the proposed method are not clear.
Also, they should report the complexity of the proposed method.

strengths:
The writing is good and also the mathematics are true.

weaknesses:
The paper's main idea is the PPCA extension containing nonlinear manifold terms. The reason for this assumption is not clear. 
But the formulation and mathematical analysis are true.
Also, its performance should be seen in real data. However the experiments on real data are not clear and also they should apply it for more general data like well-known image data sets, or at least mention the class of real data that this method can work well for them in comparison with PPCA. By this version of experiments the importance and quality of the proposed method are not clear.
Also, they should report the complexity of the proposed method.

questions:
_ experiments for real data should be done?
what is the specific class of data that this idea works good?
why they did not report the complexity?
if they report one can compare them with methods like deep factor models
</content>
</thread_object>

</thread>
<thread>
creation date:
1730709753569

thread_type:
official_review

<thread_object>
creator: reviewer_vsuw

<content>
summary:
In this work, the author proposes a Probabilistic Geometric Principal Component Analysis (PGPCA) method which can be seen as an extension of PPCA with a better description of data distributions by incorporating a nonlinear manifold geometry. A data-driven EM algorithm is also proposed to solve the PGPCA problem. Experimental results verify the performance of the PGPCA is better than that of PPCA.

strengths:
The method and algorithms are technically sound, with reasonable insights and solutions.

weaknesses:
Comparison with other nonlinear PPCA methods is not provided.

questions:
1. As some works focus on nonlinear PPCA, such as 

Lawrence, Neil, and Aapo Hyvärinen. "Probabilistic non-linear principal component analysis with Gaussian process latent variable models." Journal of Machine Learning Research 6.11 (2005).

Zhang, Jingxin, et al. "An improved mixture of probabilistic PCA for nonlinear data-driven process monitoring." IEEE transactions on cybernetics 49.1 (2017): 198-210.

how is the performance of the proposed PGPCA compared with these nonlinear PPCA methods?


2. How does PGPCA compare with other methods in terms of computational efficiency?
</content>
</thread_object>

</thread>


Metareview:
metareview:
The  paper proposes Probabilistic Geometric Principal Component Analysis (PGPCA), which is a generalization of Probabilistic PCA (PPCA) to handle the case the data is distributed around a nonlinear manifold. A data-driven EM algorithm is proposed to solve the PGPCA problem. Experimental results are provided and verify that the performance of the PGPCA is better than that of PPCA.

Note: after the discussion period, the authors changed the title of their paper (PDF file) to "Probabilistic Geometric Principal Component Analysis with Application to Neural Data" 

Reviewers generally agree that the proposed extension PGPCA of PPCA is novel and well- explained, the mathematical derivation of the EM-algorithm is concise and understandable, and the experimental results appear promising.

I do have reservations concerning the paper:

- The scope of the proposed method is limited to data of a specific type: distributed around a fixed nonlinear manifold. The method itself is still a linear method.

- The derivation of the EM-algorithm is a straightforward extension of that in PPCA. 

- The experimental results on real data are very limited, especially given the fact that the paper is aimed towards "Application to Neural Data"

However, the method could be of interest to the neuroscience community.