Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: summary: This manuscript details the inference of a probabilistic dimensionality reduction technique that is applicable to non-Gaussian data, for example spike counts. Tensor decomposition methods have been applied to neuroscience data in the past, and this paper extends this formulation to frequently seen spike data. The authors introduce a variational Bayes approach towards a Polya-Gamma augmentation of the model with negative binomial observations. 
main_review: The paper is very thoroughly written and presented. The model confers many advantages and the authors have presented convincing results. The model successfully recovers model parameters on simulated data while performing automatic relevance determination. On neural spiking data, the model class offers clear gains in variance explained and Poisson deviance explained in held-out data for R<4, when compared against canonical polyadic (CP) decomposition and generalized canonical polyadic (GCP) decomposition. 

It would help to show the results using the CP and GCP comparison methods on simulation data to better motivate the proposed method.

It would be good to comment on the comparison between CP, GCP and the proposed method for R>=4 for the neural spiking data. Figure 3 contains the metrics for a subset of the values of R; it would be great to fill this out if possible, or provide an explanation of why these values of R were chosen.

Important: please include a detailed explanation of why GCP was chosen with the Poisson observation model as opposed to the negative binomial observation model.

Please detail out GCP before the first time it is mentioned in the manuscript, as well as include a brief description of the method.

The authors mention Polya-Gamma and its acronym PG multiple times – it would be helpful to define PG once the first time they mention it, and then stick to the acronym.

limitations_and_societal_impact: Yes
review_title: null

final_score: rating: 8: Top 50% of accepted NeurIPS papers, clear accept
review_content: summary: The paper presents a probabilistic tensor decomposition model for count data with applications to neural recordings. A variational inference method using Polya-Gamma augmentation is derived to allow feasible model fitting on large neural datasets, and some theoretical results of the asymptotic behavior of the variational approximation of the PG augmentation are shown. The utility of the tensor decomposition approach is demonstrated using recordings from rodents during a sensory integration experiment.
main_review: This proposed model is a novel approach to tensor decomposition that has strong potential to be widely applicable in neuroscience (using spike train data) than standard tensor decomposition models, which use squared error instead of a likelihood on counts.
-	Additionally, it allows for a full Bayesian treatment with good use of the Polya-Gamma augmentation in variational inference, rather than slower sampling. Advantages of this approach are demonstrated by, for example, performing rank selection using an ARD prior.

The presentation is generally clear, and appropriate comparisons to existing methods are given.
However, a couple points could improve the clarity of the results and utility of the approach:
-	The real data example did not include much discussion about what the tensor composition showed. Although comparisons to CP and GCP decompositions are provided in the supplementary, it’s not immediately clear to the reader what is gained in this case.
-	For comparing the GCP and CP to the probabilistic model (e.g., fig 3), it would be helpful to include the number of parameters in each model. I think this would help mainly because of the offset tensor included in the probabilistic model.


limitations_and_societal_impact: Yes.
review_title: null

final_score: rating: 8: Top 50% of accepted NeurIPS papers, clear accept
review_content: summary: In the paper "Probabilistic Tensor Decomposition of Neural Population Spiking Activity" the authors propose a probabilistic model for canonical polyadic tensor decomposition and apply it to neural data. The model assumes negative binomial observations and is used with variational Bayesian inference. The method is evaluated on synthetic data and on multisensory integration activity recorded from mice.
main_review: Originality:
As noted by the authors, Rai et al. IJCAI 2015 used the Polya-Gamma approach as well in the tensor decomposition setting but did not achieve posterior variance estimates. For all I know, the proposed method enabling to probabilistically determine the rank and estimate variance in the CP decomposition setting is novel. The method makes use of a negative binomial observation model which is appropriate for many neuroscience datasets.

Quality:
The quality of the work is outstanding. The method is derived in a sensible and rigorous way. However, I did not check all equations.
The evaluation includes both synthetic data with known ground truth as well as data recorded from mice illustrating the utility of the approach on real data as well as yielding interesting results with regard to multisensory integration factors.
I found it surprising that the proposed method outperformed the baseline (G)CP decompositions on variance explained and Poisson deviance explained even though these are what the loss functions of the baseline models optimize for. In the Discussion the authors speculate that this is due to the constrained offset tensor. Couldn't this be easily tested with a variant of the proposed method without the offset tensor?

Clarity:
The manuscript is very well written. The work is well motivated and presented with a useful illustration (Figure 1). All figures are informative and can be understood easily.

Significance:
Given datasets of growing sizes in neuroscience, tensor decompositions are promising tools to find easy to interpret patterns in these datasets. A probabilistic model that can be used to quantify uncertainty and to determine the rank automatically is very useful in this context.
limitations_and_societal_impact: The author discuss limitations with regard to uncertainty estimates in the Discussion. The limitations paragraph could have been more extensive as the authors did not use all of the available space. I would have welcomed a discussion on the negative binomial assumption given that sometimes neuronal data have a fano factor less than 1.
review_title: null

Meta-review: The authors develop tensor decompositions for count data, like neural spike trains, as well as a variational inference approach based on the Pólya-gamma augmentation. The reviewers and I agree that the paper is novel, well-presented, and of broad interest to the computational and statistical neuroscience community at NeurIPS.
Decision: Accept (Spotlight)