Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper introduces a latent variable model for both generating and clustering neural activity. In simulated datasets, they study the performance of the model and its ability to recover underlying ensembles in a synthetic and real-world toroid task.
Overall, the paper provides an interesting model and a strong set of different evaluations of the method, across different grid cell datasets and a synthetic toroid task where they set up an ensemble detection task. However, there are some concerns about the generalization of the approach to different tasks and neural datasets. 



strength_and_weaknesses: Strengths:
+ The method seems to provide some good advances in both its generative capability as well as the ability to predict ensembles.

+ They introduce a new toroid task and measures of ensemble detection in this case.


Weaknesses:
- The authors compare three variants of their general approach - a shared, no-shared feature model, and a heat kernel model. While there is some discussion of how the different models provide varying benefits in different regimes (for synthetic data), the differences between the models and underlying assumptions is not entirely clear. In the experiments it seems that the shared features model doesn't give improvements across the board -- but isn't this is the core motivation of the work? 

E.g., What is the difference between the non-shared model (faeLVM-n) and the geometric LVM model (mGP)? It would be helpful to have a more clear description of the ablations and other models included in the comparisons.

- In the experiments, the number of clusters is relatively low (2-3) and there isn’t much discussion about how the number of clusters is selected (or the regularization is performed to encourage soft clustering of neurons). How does the method work when there are more clusters or larger amounts of overlap between ensembles? How do you select the number of ensembles or clusters and/or regularization in the soft clustering assignment?

- The applications and datasets that are examined are somewhat limited and it's unclear how well it will work in different tasks that don’t have strong geometry and when the feature subspaces are non-overlapping. In particular, is it possible to use their method in conditions where ensembles are not as separable or more entangled?

- The set of baselines are limited. There are a number of other methods for latent variable modeling in neural data analysis, including many cited in the work and in the Neural Latents Benchmark cited in the evaluation section. It would be interesting to understand how their model compares to some of these other methods and in other datasets that don’t have clear toroid structure or where neurons are separated as cleanly as in the datasets tested here.

- For the ensemble detection method, what is the performance for the other variants of your approach? What about feature sharing? It seems this would be the natural model used for these experiments (and not just the -b model).
clarity,_quality,_novelty_and_reproducibility: The paper is relatively well written and the approach is clear. Some details on the different variants of the model could be elaborated upon to better understand and interpret the results. 
summary_of_the_review: This paper provides a new latent variable model for neural population activity that can leverage shared features across different neurons to learn population-level representations and also detect ensembles (neurons that use shared features). The model provides an interesting blend between population-level analysis and more interpretable neuron-level measures of tuning. 

In empirical evaluations, the authors show that their model can provide good predictions of held out neural activity and thus serves as a good generative model, and also can recover underlying ensembles in the data. However, it is unclear how general the model is, how much they rely on simplified tasks where the subgroups are orthogonal or well separated, and how much the geometry of the underlying task matters. The relatively simple conditions in which the model is tested makes it hard to assess the generalization of the approach to different tasks or conditions, or even larger numbers of clusters.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The authors propose a new latent variable model for neural data (faeLVM). The faeLVM is a generative model that combines the insights provided by tuning curves with the interpretability of low-dimensional latent variables. Specifically, faeLVM models a population of neurons are a collection of ensembles, where neurons in an ensemble are sensitive to a subset of the latent variables and share the functional form of the tuning curve, which the authors call feature sharing. For training, the authors use the VAE framework and utilize a variational posterior and optimize the ELBO. The main point of this paper, in my mind, is to make VAE-style models interpretable by making the decoder portion very simple, while still allowing for an arbitrarily complex encoder.
strength_and_weaknesses: The motivation behind this paper is very nice. The idea is similar to Manifold GPLVMs (Jensen et al. 2020), but in many ways the approach taken in this paper is simpler to implement and conceptualize. I really like the idea of having flexible and interpretable "deep" latent variable models for neuroscience.

The main weakness of this paper is that it uses what I would consider "easy" experimental datasets to make the case for simple tuning-curve-like decoders. For example, the grid cell data used pre-selected, non-conjunctive grid cells from MEC. This is a very special subset of the full MEC population, and it would be much more compelling if the method worked well on the raw data. At a minimum, the authors should discuss this limitation and cite [Hardcastle et al. 2017](https://doi.org/10.1016/j.neuron.2017.03.025), which characterizes the messy, conjunctive coding in the full MEC population.

In short, I think the feature sharing idea proposed by the authors can be useful on certain neural datasets, but it will be a far from universal component you'd like to build into the model. I have my doubts about whether this will become a commonly used model in the neuroscience literature (outside of Neuro-ML overlapping conference venues).

I would also like to see the paper edited for clarity, as discussed below.
clarity,_quality,_novelty_and_reproducibility: - In equation 1 and 2, the use of subscripts and superscripts is very confusing and don’t match up with notation previously used. For instance, above equation 1 the authors state “… helps keeping the equations the same when working with multiple latents (corresponding to multiple ensembles), i.e., z = {z_1, …, z_k}” implying that k is the total number of ensembles. But in equation 1, k is being used to denote the dimension of z_j. It would help if the authors explicitly defined lower and upper limits on the product symbols.
- In section 3, the authors state that they are using a von Mises distribution as their variational posterior and prior but this seems to be at odds with section A.1 in the appendix. Specifically, from the appendix the authors seem to be using the ReLie trick from Falorsi et al., 2019. While the ReLie trick allows one to produce reparametrizable samples from S(n), this does *not* correspond to samples from the von Mises distribution (at least to my understanding).
    - **Here is a suggestion.** Use a projected normal distribution ([see Presnell et al. 1998](https://doi.org/10.1080/01621459.1998.10473768)) instead of von Mises. Unless I'm misunderstanding something, it should be fine with the reparameterization trick.
- I can’t seem to find the dimension of z is used for each of the experiments. Moreover, it isn’t also isn’t clear what latents follow a normal distribution versus a distribution on S(1). This is compounded with the fact that it also isn’t clear whether there is a subset of latents shared across the ensembles. This makes the experiments very hard to reproduce.
- In section 4.3, why isn’t the rate prediction NLL shown for the VAE case? While I understand that mGP isn’t shown because they use a Gaussian likelihood, VAEs are flexible and can be used with any likelihood distribution. This is important, especially since the VAEs seem to perform on par with faeLVM-s (with much less variability) and outperforms faeLVM-b.
- I don't fully understand how the one-hot encoding is achieved in the ensemble selection step of the network. It feels like there should be a hyperparameter controlling the temperature / sparseness of the softmax operation. Is there guidance for how users can tune up or down this sparsity?
summary_of_the_review: Overall, I like the basic idea of this paper. VAE decoders are too often treated as black boxes in scientific applications where interpretability is centrally important. I think this paper has some flaws and the empirical demonstrations are in idealized datasets. Nonetheless I am giving it the benefit of the doubt and recommending it as a borderline accept for now, pending the authors willingness to address some of the comments above.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The authors propose a latent variable model that extracts meaningful low-dimensional representations from large scale neural recordings. The proposed method assumes Gaussian-like tuning curves to the latent variable on the neurons which share an ensemble-wise basis. The spherical variational posterior captures the toroidal manifold of variables such as head direction that is encoded in the neural activity.
strength_and_weaknesses: Strength:
Using biologically plausible tuning curves
Spherical variational posterior captures toroidal manifold
Identifying neuron ensembles

Weaknesses:
Strong assumptions such as tuning curve and vMF posterior limit the scope of the proposed method. Most LVMs for spike trains aim at explaining most of the shared variability of the data using a low-dimensional representation. On the contrary, the demonstration of this work only shows it perfectly fits to coding toroidal behavioral variables. The vMF posterior forces the latent variable to be toroidal and mimicking Gaussian-shape tuning curve to head direction of the target neurons also forces the latent variable to be close the head direction. It is fine as   we know the toroidal head direction and the shape of tuning curves beforehand, but this demonstration makes the method seem less nontrivial. 
 
clarity,_quality,_novelty_and_reproducibility: The writing is mostly clear. Some need clarification.
- $x_i$ is the firing rate at the beginning of the background, and a deterministic function of the latent variable $z$. However, it seems to be stochastic according to (3).
- Why the neuron firing rate has no bias, or one of g's is constant?
- How are the ensembles determined?

The work shows a decent quality.
- Better to compare the reconstruction error for all methods. It kind of reflects how much variability is captured. Though the latent variable is perfectly encoding certain variable, it might be missing other variability in the neural activity. Since you mentioned Pei2021, why not use bps, it's available for non-Poisson likelihood too.
- Depending on the nonlinearity from latent variable to firing rate, the models might have equivalent latent variable subject to a linear/nonlinear transformation.
- vMF posterior guarantees the ring or torus. What if a non-toroidal variable is encoded? Better to evaluate it in the case of model mismatch.

The combination and use of the methods are novel.
summary_of_the_review: The authors propose a latent variable model that extracts meaningful low-dimensional representations from large scale neural recordings. The proposed method assumes Gaussian-like tuning curves to the latent variable on the neurons which share an ensemble-wise basis. The spherical variational posterior captures the toroidal manifold of variables such as head direction that is encoded in the neural activity. Strong assumptions such as tuning curve and vMF posterior do help the inference to get latent variable desirably matches the behavior. They also limit the generalization. What more do we know from it about the data than an encoding model with the same tuning curves?
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: This paper introduces a new latent variable modelling approach for neural spike trains that extends the normal iid latent prior to also latent distributions on a torus and extends the generative model to use tuning curves. 

All reviewers liked the paper but also had a few reservations regarding the general applicability.

A good solid paper. Accept.
justification_for_why_not_higher_score: Reviewers not overentusastic. 
justification_for_why_not_lower_score: The paper introduces some new features in the VAE framework that might potentially be useful in other settings as well.
Decision: Accept: poster