Paper:
Neurons as Monte Carlo Samplers: Bayesian Inference and Learning in Spiking Networks
We propose a spiking network model capable of performing both approximate inference and learning for any hidden Markov model. The lower layer sensory neurons detect noisy measurements of hidden world states. The higher layer neurons with recurrent connections infer a posterior distribution over world states from spike trains generated by sensory neurons. We show how such a neuronal network with synaptic plasticity can implement a form of Bayesian inference similar to Monte Carlo methods such as particle filtering. Each spike in the population of inference neurons represents a sample of a particular hidden world state. The spiking activity across the neural population approximates the posterior distribution of hidden state. The model provides a functional explanation for the Poissonlike noise commonly observed in cortical responses. Uncertainties in spike times provide the necessary variability for sampling during inference. Unlike previous models, the hidden world state is not observed by the sensory neurons, and the temporal dynamics of the hidden state is unknown. We demonstrate how such networks can sequentially learn hidden Markov models using a spike-timing dependent Hebbian learning rule and achieve power-law convergence rates.

Review:
1070: Neurons as Monte Carlo Samplers: Bayesian Inference and Learning in Spiking NetworksThe authors study a possible neuronal implementation of learning and inference in a hidden Markov model. As a major novelty, the authors propose that the stochasticity of synaptic transmission is directly involved in the implementation of stochasticity necessary for Monte Carlo sampling. The neurons used throughout the paper are binary threshold units and not spiking neurons. These binary neurons are able to provide useful insights into how a neuronal network may solve computational problems, but it is important to distinguish between implementations using binary units and spiking neurons. The authors include a short section about spike-based implementation in the appendix, but they do not demonstrate that the spike based implementation is able to perform the same tasks with similar performance. For example, the short term plasticity they propose in the supplementary material to prevent firing after multiple spikes from the recurrent or feed-forward inputs, is not going to help, as STP acts between spikes arriving from at a single synapse not on inputs arriving from a pool of neurons.Inference: The authors write in the introduction that previous work has been limited to binary word states. However, any discrete state space can be represented by a set of binary variables (just as they do in Fig 2b), hence its is not entirely clear how the paper differs from those previous studies in this aspect.From the derivation above Eq. 5. and the that C_W~L it follows that the release probability scales with 1/L. However, the release probability in cortical synapses is around 0.5. The authors need to comment on this difference. During the proof of Eq. 6 the authors take the limit that the spiking activity is sparse. However, it is not clear why this limit is relevant here, especially since sampling is more efficient in the high firing rate limit. The prediction for the Poisson variability also follows from the sparsity assumption and it should be confirmed by simulations.On Fig. 2a the authors should compare the approximate posterior inferred by the model with the true posterior and not only with the data. They should use a quantitative measure to test the performance of the network (e.g., KL-divergence). They show marginal An interesting prediction of the theory would be that the sum of the release probabilities at the outgoing synapses of a neuron is constant (C_W) while the same sum for the incoming synapses covaries with the marginal state probability.Learning:The learning rule: Eq. 11 and 12 should be replaced by Eq. 31 and 35 of the appendix. Learning is non-local, since it depends on the number of spikes in the (pre and postsynaptic) population (n) and not on the actual spiking of the neuron (s).  It is not clear from the data provided whether the learning rule converges to the right value in the case when both the emissions and the hidden dynamics are learned together.