Paper:
Low-dimensional models of neural population activity in sensory cortical circuits
Neural responses in visual cortex are influenced by visual stimuli and by ongoing spiking activity in local circuits. An important challenge in computational neuroscience is to develop models that can account for both of these features in large multi-neuron recordings and to reveal how stimulus representations interact with and depend on cortical dynamics. Here we introduce a statistical model of neural population activity that integrates a nonlinear receptive field model with a latent dynamical model of ongoing cortical activity. This model captures temporal dynamics and correlations due to shared stimulus drive as well as common noise. Moreover, because the nonlinear stimulus inputs are mixed by the ongoing dynamics, the model can account for a multiple idiosyncratic receptive field shapes with a small number of nonlinear inputs to a low-dimensional dynamical model. We introduce a fast estimation method using online expectation maximization with Laplace approximations, for which inference scales linearly in both population size and recording duration. We test this model to multi-channel recordings from primary visual cortex and show that it accounts for neural tuning properties as well as cross-neural correlations.

Review:
One of the chief concerns of systems neuroscientists is characterising how individual neurons respond to sensory stimuli. Since the stimulus space is often huge, data is always limited, and neurons are fundamentally noisy, the statistical challenges involved with this characterisation have spurned a vibrant field of computational neuroscience. This paper considers a particular form of this task, where recordings are made of local populations of neurons, at some intermediate point in the processing hierarchy in the brain (e.g. neurons within a cortical column, within a primary cortical sensory area). Such recordings are very common already, and also are growing in number and fidelity. The paper argues that under these conditions, an assumption can often be made that the neurons under study are receiving inputs from a smaller number of common channels, which are both stimulus-driven and subject to recurrent dynamics. The authors proceed to formalise this as a statistically-reasoned generative model for the observed neural responses (as spike counts), and present a Laplace/EM inference scheme to invert it given population data. The authors validate the inference on a simulated data set, and then fit it to V1 data.Overall, I think this is a well-motivated problem, and the solution the authors offer is both novel and interesting. The paper is clear and well-written, and makes a useful contribution to the field. There are a few issues here and there, which I list below.More importantly, I think there's some weakness in the scientific interpretations of these results that the authors could clean up. The construction the paper champions is essentially a beefed-up subunit model. Subunit models have a long and rich history in systems neuroscience, starting at least as early as Barlow & Levick (1964) or Hochstein & Shapley (1976) (perhaps we should include Hubel & Wiesel 1959). These are also having renewed interest as hierarchical models of feedforward processing, e.g. Mineault et al (2012). The first stage of the proposed model in Figure 1 is essentially a collection of nonlinear subunits. The authors do flag subunits in the second sentence of the introduction, but this link (and terminology) is dropped beyond this point. Now, of course, the beef this paper adds is important and useful, and I am a fan. However, I think that the systems neuroscience community would be more willing to engage with this work and build upon it if the subunit idea were carried through, especially because it captures an intuition about the feedforward processing circuits in the brain. And, in the absence of an argument to the contrary, I think it would be worthwhile to use the terminology of subunits to refer to this first stage within the model. Some of these subunits, of course, would not be carrying stimulus-driven information, only latent dynamical signals; this shouldn't be a big deal.Given that, I think there is some interpretation that needs to be made of the results of the simulation (Fig 2A-B especially), and the real data (Fig 4F especially). In the simulation, the ground truth (and recovered) subunits look like simple cells; in the real recordings, the recovered subunit space looks more like LGN inputs (albeit without a visible surround). As much as I rotate these latter recovered features in my head to get around the identifiability issue, I can't make the two examples match. By the time Section 4 has come around, the authors have set everything in place for a big reveal of simple-cell subunits, and it's rather disappointing that the results don't deliver. Moreover, it's not clear how the recovered features would generate the type and diversity of V1 receptive fields we're expecting. There are many potential reasons why this might be the case (e.g. sampling biases; all the units were simple), but the authors are in a better position than I to provide a reasonable explanation of this. To make things stranger, the first paragraph of the discussion seems to claim that the model can capture both simple cells and complex cells, and that it does. This was not the impression I came away with from Figure 4. Ultimately, perhaps the paper is principly about a model, but V1 is probably the most well-understood model testbed for neural systems identification, and there are expectations for the results here. If no explanation is given to intuitively validate the recovered features, readers will conclude that it doesn't recover what's expected here, so it won't work elsewhere.Quality: The paper is technically sound, with a couple of minor typographical errors listed below. Aside from the larger issue above, one lingering question I had was how to choose the number of dimensions in the first layer. The analysis in Fig. 3 (and surrounding text) seems like the right place to do this, but as far as I can tell, the number of stimulus-driven and stimulus-independent subunits were fixed at the ground truth. There should be some discussion of what happens when this is underestimated or overestimated. Figure 3 does not provide good enough evidence, as this assumes the ground truth (and the trends are hard to see in 3A).Minor errors:- eq. (2) and line 100-2: should $\vec{s}_t$ be the spike count vector $\vec{y}_t$, for consistency with eq. (3), and the typically implementation of the spike feedback kernel? Then this vector should also be integer-valued not real-valued. - line 183: C/D/d should be boldfaced (as matrix/matrix/vector), not scalar- eqn (7), LHS: the $\vec{w}_t^T$ should be a $\vec{h}_t^T$. Still, I think these equations could be omitted since the derivatives are pretty simple. Clarity: In general, the paper is written clearly and organised well. There were a few points throughout the text where I felt some extra clarification would make it read better:- eqn (2) and below: $\vec{D} \cdot \vec{s}_t$ -- I would suggest using a convolution symbol $\ast$ in place of $\cdot$ to convey its intuition. Also the constant $\vec{d}$ is not annotated.- line 115-117: There's some ambiguity about what refers to individual neurons, what refers to the local (recorded/columnar) subpopulation, and what refers to all of V1. There's at least one way of misinterpreting what is being said here. Individual neurons may care only about a stimulus subspace (hence STC). A local subpopulation will care about a larger stimulus subpsace than any one neuron (though not proportionally larger). The whole of V1 probably cares about a substantial portion of the whole stimulus space. The important aspect of *this* model is that it learns a basis for the local subpopulation's responses, which is assumed to be reasonably small. This is captured in the introduction to the paper, but is obscured here.- line 126/127, and subsequent: I found it confusing to talk about $\vec{B}$ as the concatenation of all /nonlinearity/ parameters, when $f$ could be a linear function. This is an instance where /subunit/ parameters would work better.- line 146: make this "where the parameters B = {...} includes"- line 166: "one infers the distribution of trajectories," -- insert $\vec{x}_t$ here. Likewise, start of line 168, "estimates" -- insert a $\vec{B}$ here. - line 172/173: the covariances $\vec{Q}_t$ of the latent states. Better to call this $\vec{\Lambda}_t$ or $\vec{\Sigma}_t$ or something like that, to avoid conflation with the covariance of epsilon_t. Line 181 is currently particularly confusing with both Q_t and Q.- further to that, I wasn't entirely clear what the posterior covariances $\vec{Q}_t$ should look like. This seems like it could be of some importance with the minibatch procedure. Also is it correct that the posterior covariance on the latent states only affects inference on the dynamical system parameters, and not on the subunit/nonlinearity parameters? Beyond this, I didn't feel like I got a good sense of how the information was combined across minibatches to update both sets of parameters in the M-step.- Fig 2C: wasn't clear exactly how the noise correlations were estimated without repeats (as for 4D).- Fig 2E: some info is necessary on how to interpret this plot, and what its relevance is. Same again for 4D and 5D. This appears to be playing the role of a validating result, but for Fig 4D and 5D I have no idea why this conveys as much.- line 272: I was amused by the precision in the computation time, but I wasn't sure how to extrapolate from this anecdote.Originality: tick, but some work needed to link to subunit literature (see above).Significance: tick, but some work needed to link to subunit literature (see above).