Paper:
Mental Sampling in Multimodal Representations
Both resources in the natural environment and concepts in a semantic space are distributed "patchily", with large gaps in between the patches. To describe people's internal and external foraging behavior, various random walk models have been proposed. In particular, internal foraging has been modeled as sampling: in order to gather relevant information for making a decision, people draw samples from a mental representation using random-walk algorithms such as Markov chain Monte Carlo (MCMC). However, two common empirical observations argue against people using simple sampling algorithms such as MCMC for internal foraging. First, the distance between samples is often best described by a Lévy flight distribution: the probability of the distance between two successive locations follows a power-law on the distances. Second, humans and other animals produce long-range, slowly decaying autocorrelations characterized as 1/f -like fluctuations, instead of the 1/f 2 fluctuations produced by random walks. We propose that mental sampling is not done by simple MCMC, but is instead adapted to multimodal representations and is implemented by Metropolis-coupled Markov chain Monte Carlo (MC 3 ), one of the first algorithms developed for sampling from multimodal distributions. MC 3 involves running multiple Markov chains in parallel but with target distributions of different temperatures, and it swaps the states of the chains whenever a better location is found. Heated chains more readily traverse valleys in the probability landscape to propose moves to far-away peaks, while the colder chains make the local steps that explore the current peak or patch. We show that MC 3 generates distances between successive samples that follow a Lévy flight distribution and produce 1/f -like autocorrelations, providing a single mechanistic account of these two puzzling empirical phenomena of internal foraging.

Review:
One of the current speculative hypotheses in cognition is that the brain performs approximate Bayesian inference via some form of sampling algorithm. Based on this assumption, this paper explores which kind of Monte Carlo algorithm the brain might be using. In particular, previous work has proposed direct sampling (DS) from the posterior distribution, or random-walk MCMC. However, these two algorithms are unable to explain some empirically observed features of mental representations, such as the power law seen in the distance between consecutive, distinct “samples” (such as responses in a semantic fluency task) or, equivalently under certain assumptions, of the distribution of inter-response intervals. Also, autocorrelation of mental samples exhibit a specific “1/f” pattern. This paper argues that another type of MCMC algorithm, that is MC3 aka parallel tempering, which is a MCMC method designed to deal with multimodal, patchy posteriors (hence the “foraging” analogy), is instead able to explain these features. The authors demonstrate the better empirical validity of MC3 -- mostly as a proof of concept -- with a toy example and with real data obtained in a simple novel experiment.  Comments:  Quality: The paper is technically sound and, if we accept the whole “mental sampling” paradigm, presents an interesting hypothesis for how sampling is performed (i.e., via parallel processes -- chains -- that operate at different “temperatures”, and occasionally swap information).  The current work is mostly a proof of concept in that the authors show general quantitative agreement with the data for some specific algorithm choices (e.g., temperature range, number of chains, proposal distributions). I think that a much broader exploration of the (hyper)parameter space is beyond the scope of this work (and perhaps pointless, since this is mostly a proof of existence), but it would be reassuring to see that these details do not substantially affect the results, at least for a couple of examples. Given the simplicity of the analysis, it should be straightforward to try out a few other algorithmic configurations. Importantly, the authors already show that RwM still does not reproduce features of the data when changing the proposal distribution to a heavy-tailed distribution, probably the most relevant control. However, a couple more examples would not hurt.  Clarity: The paper is well-written and easy to follow throughout. Only found a couple of typos: line 111: follows → follow line 131: MCMC, was → MCMC, which was  Originality: This work extends previous computational work in the cognitive sciences that proposed sampling (and specifically MCMC) as an algorithm used by the brain for approximating Bayesian inference. The novelty of this proposal is that it explains a couple of important features of observed data (Lévy flights and 1/f noise) via a MCMC method which was not previously considered as a model for cognition, as far as I know.  Significance: There has been a trend in the past years of describing any chosen aspect of cognition via some specific modern (or less modern) machine learning technique, which is somewhat suspicious given that as humans we always had the tendency to describe the functioning of the brain -- once we started to understand its relevance for cognition -- with comparisons to the latest, fanciest engineering advancement. However, the current proposal is interesting in that some of the algorithmic features (e.g., the existence of different processes/chains, the associated temperatures) might be subject to direct experimental investigation, perhaps making this hypothesis testable beyond some qualitative agreement and proof of concept.  In conclusion, this is a nice conference paper which presents an interesting idea and some preliminary support for it.  After Author Feedback: Thank you for the clarifications. I find positive that the authors are going to share their code (I hope all the code to reproduce the experiments in the paper), and increases my belief in reproducibility of their results.