Modeling correlations in spontaneous activity of visual cortex with centered Gaussian-binary deep Boltzmann machines
Authors: nan wang, Laurenz Wiskott, Dirk Jancke
ICLR 2014

Abstract: Spontaneous cortical activity -- the ongoing cortical activities in absence of sensory input -- are considered to play a vital role in many aspects of both normal brain functions and mental dysfunctions. We present a centered Gaussian-binary deep Boltzmann machine (GDBM) for modeling the activity in visual cortex and relate the random sampling in DBMs to the spontaneous cortical activity. After training on natural image patches, the proposed model is able to learn the filters similar to the receptive fields of simple cells in V1. Furthermore, we show that the samples collected from random sampling in the centered GDBMs encompass similar activity patterns as found in the spontaneous cortical activity of the visual cortex. Specifically, filters having the same orientation preference tend to be active together during random sampling. Our work demonstrates the homeostasis learned by the centered GDBM and its potential for modeling visual cortical activity. Besides, the results support the hypothesis that the homeostatic mechanism exists in the cortex.


Peer review:
[Review]:
The authors use a Gaussian-binary deep Boltzmann machine (GDBM) to model aspects of visual cortex. Training of the GDBM involves the centering trick of [8]. The comparison to visual cortex is in terms of learned receptive field properties, and by reproducing experimentally observed properties of activity in V1 as reported by [1]. In particular, these findings relate spontaneous activity in the absence of external stimulation to evoked activity.

On the positive side, I think the issue of the nature of spontaneous activity is interesting, and the authors put effort into reproducing the experimental findings of [1]. On the negative side, the significance of the main contributions seems lacking to me, and the authors need to motivate better why their work is important or relevant. Quality and clarity need to be improved in several points as well.


Details:

To expand on the above, I'll discuss three main points: 1) The centered GDBM. 2) Reproducing aspects of visual cortex. 3) The connection to homeostasis.

1) I wouldn't see this part as a major contribution of the paper. The centering trick was originally applied to a fully binary DBM. Applying the same trick to the GDBM (which only differs in having a Gaussian visible layer) seems a very natural thing to do. Moreover, there is no further analysis on the efficacy of the centering trick. The authors say that centering makes the GDBM easy to train compared to [15], but they don't actually evaluate the performance of the GDBM (other than comparing it to biological phenomenology), and only apply it to image patches. In [15], the GDBM was trained on images of faces and evaluated in terms of filling in missing parts of the image. Hence, it is not clear whether centering makes the more complicated training of [15] obsolete, as suggested by the authors.

Also, when it comes to clarity: given that the authors emphasize the importance of centering, they need to better explain what it is, why it works, and how the centering parameters are computed (the latter is only described in the algorithm float). These things are unclear to the reader unless they look at the reference.


2) Reproducing aspects of visual cortex is the main contribution of the paper. First, the learned receptive fields are suggested to qualitatively resemble those in V1 and V2. Learning V1-like Gabor filters is of course a quite common result nowadays. I don't think it's possible to conclude that the model captures receptive field properties V2 well, just from looking at Figure 1b.

Hence, the main contribution here is the analysis of activity. I think the results are fine and somewhat interesting, though not surprising. What is missing is better motivating/explaining why these results are interesting/relevant. Sure, the GDBM reproduces certain experimental findings. But have we learned something new about the brain? Is the GDBM particularly well suited as a general model of visual cortex? Does the model make predictions? What about alternative, perhaps simpler models that could have been used instead? Etc.

Also, are there related models or theoretical approaches to spontaneous activity? For example, this comes to mind:

Berkes, P., Orbán, G., Lengyel, M., & Fiser, J. (2011). Spontaneous cortical activity reveals hallmarks of an optimal internal model of the environment. Science (New York, N.Y.), 331(6013), 83–7. doi:10.1126/science.1195870

As for clarity: when collecting the spontaneous frames, presumably you are sampling from the full model distribution P(x,y,z) via Gibbs-sampling, with all layers unclamped, correct? Because writing that samples were collected from P(y|x,z) seems to suggest that you clamp to a specific x and z and collect multiple samples from the same conditional (not just as a step during Gibbs sampling). 


3) Lastly, the connection between homeostasis and the author's model is unclear. The authors mention homeostasis in the abstract, introduction and discussion, but do not explain homeostasis or how their results relate to it specifically. This needs to be explained better, especially to this audience. Personally, I know the literature somewhat (e.g. [4]), but am nevertheless unclear about what exactly the authors intend to say. 
Sentences such as 'we are able to make the model learn the homeostasis from natural image patches' are unclear.

When I first read the paper, I thought the authors were referring to the centering trick as something that can be understood as a homeostatic mechanism (i.e., a neuronal mechanism that maintains a certain average firing rate, see [4], [6,7]), which would make sense to me. However, on further reading it seems to me that the authors refer to the fact that spontaneous activity resembles evoked activity as an aspect of homeostasis? Why? For comparison, [6,7] clamped the input to empty images (to simulate blindness), and had an active homeostatic mechanism at play that led to spontaneous activity resembling evoked activity. In the authors' paper, spontaneous activity resembles evoked activity simply because the former is taken to be sampled from the unconditioned model distribution..? I'm not sure where homeostasis, i.e. being subject to an active self-regularity mechanism, comes into play at this point.

(As an aside, I don't think the Friston reference [3] clears up what the authors' notion of homeostasis is, in particular as Friston talks about states of agents in their environments. Frankly, Friston's theoretical claims are often unclear to me, to say the least, in particular when it comes to mixing together internal models in the brain, and methods that should apply to the latter such as variational inference, and external probabilistic descriptions of agents and environments. Either way, if the authors would like to make a connection to Friston's theories in particular, then that connection should be explained better. Generative/predictive brain models and homeostatic mechanisms per se are not exclusive to Friston's theory.)


Further comments:

* Abstract, 'Spontaneous cortical activity [...] are' -> is

* 2.1 first para, 'consisted' -> consisting

* Not sure why x,y,z are sometimes capitalized, sometimes not.

* 3.1 'sparse DBNs show worse match to the biological findings as reported in [1]. A quantitative comparison between centered GDBMs and sparse DBNs is still open for future studies.' Where was it shown to be a worse match then?

* Figure 2: shouldn't the angles (figure titles) go from 0 to 180?


In conclusion, I think this work could potentially be interesting, but in its current form quality and clarity are somewhat lacking and significance is not quite clear.