PenguinMan/ARXIV huggingface dataset:
Novelty is not surprise: Human exploratory and adaptive behavior in sequential decision-making
Authors: He A Xu, Alireza Modirshanechi, Marco P Lehmann, Wulfram Gerstner, Michael H Herzog
Journal: PLOS Computational Biology
Accepted: 2021-05-12T00:00:00Z


Classic reinforcement learning (RL) theories cannot explain human behavior in the absence of external reward or when the environment changes. Here, we employ a deep sequential decision-making paradigm with sparse reward and abrupt environmental changes. To explain the behavior of human participants in these environments, we show that RL theories need to include surprise and novelty, each with a distinct role. While novelty drives exploration before the first encounter of a reward, surprise increases the rate of learning of a world-model as well as of model-free action-values. Even though the world-model is available for model-based RL, we find that human decisions are dominated by model-free action choices. The world-model is only marginally used for planning, but it is important to detect surprising events. Our theory predicts human action choices with high probability and allows us to dissociate surprise, novelty, and reward in EEG signals.
Author summary: Humans like to explore their environment: children play with toys, tourists explore touristic sites, and readers start a new book. Exploration is useful to build knowledge about the world in the form of a ‘world-model’. However, since the world is complex and changing, the learned world-model is sometimes wrong: if so, the feeling of surprise arises. Here, we distinguish surprise from novelty; we show that humans use surprise as a signal to decide when to adapt their behavior, while they use novelty to decide where and what to explore—to eventually develop an improved world-model. Intuitively, it seems obvious to use world-models to plan future actions. However, we show that in a complex and changing environment where planning needs heavy computations, participants rarely follow an explicit plan and take their actions mainly by shaping habits. Importantly, we show that the main role of their world-model is to signal when to be surprised and, hence, when to adapt their habits. In summary, our results show how surprise and novelty interact with human reinforcement learning, contribute to human adaptive and exploratory behavior, and correlate with EEG signals.
Adult medial habenula neurons require GDNF receptor GFRα1 for synaptic stability and function
Authors: Diana Fernández-Suárez, Favio A Krapacher, Katarzyna Pietrajtis, Annika Andersson, Lilian Kisiswa, Alvaro Carrier-Ruiz, Marco A Diana, Carlos F Ibáñez
Journal: PLOS Biology
Accepted: 2021-10-05T00:00:00Z


The medial habenula (mHb) is an understudied small brain nucleus linking forebrain and midbrain structures controlling anxiety and fear behaviors. The mechanisms that maintain the structural and functional integrity of mHb neurons and their synapses remain unknown. Using spatiotemporally controlled Cre-mediated recombination in adult mice, we found that the glial cell–derived neurotrophic factor receptor alpha 1 (GFRα1) is required in adult mHb neurons for synaptic stability and function. mHb neurons express some of the highest levels of GFRα1 in the mouse brain, and acute ablation of GFRα1 results in loss of septohabenular and habenulointerpeduncular glutamatergic synapses, with the remaining synapses displaying reduced numbers of presynaptic vesicles. Chemo- and optogenetic studies in mice lacking GFRα1 revealed impaired circuit connectivity, reduced AMPA receptor postsynaptic currents, and abnormally low rectification index (R.I.) of AMPARs, suggesting reduced Ca2+ permeability. Further biochemical and proximity ligation assay (PLA) studies defined the presence of GluA1/GluA2 (Ca2+ impermeable) as well as GluA1/GluA4 (Ca2+ permeable) AMPAR complexes in mHb neurons, as well as clear differences in the levels and association of AMPAR subunits with mHb neurons lacking GFRα1. Finally, acute loss of GFRα1 in adult mHb neurons reduced anxiety-like behavior and potentiated context-based fear responses, phenocopying the effects of lesions to septal projections to the mHb. These results uncover an unexpected function for GFRα1 in the maintenance and function of adult glutamatergic synapses and reveal a potential new mechanism for regulating synaptic plasticity in the septohabenulointerpeduncular pathway and attuning of anxiety and fear behaviors.

The GPI-anchored protein GFRα1 is highly expressed in the habenula, but its role in the adult nervous system remains unknown. This study explores the role of GFRα1 in habenular neuron function, revealing that it regulates the maintenance of adult synapses in the septohabenular pathway, AMPA-mediated glutamatergic signaling, and fear and anxiety behaviors.
A multistudy analysis reveals that evoked pain intensity representation is distributed across brain systems
Authors: Bogdan Petre, Philip Kragel, Lauren Y Atlas, Stephan Geuter, Marieke Jepma, Leonie Koban, Anjali Krishnan, Marina Lopez-Sola, Elizabeth A Reynolds Losin, Mathieu Roy, Choong-Wan Woo, Tor D Wager
Journal: PLOS Biology
Accepted: 2022-04-07T00:00:00Z


Information is coded in the brain at multiple anatomical scales: locally, distributed across regions and networks, and globally. For pain, the scale of representation has not been formally tested, and quantitative comparisons of pain representations across regions and networks are lacking. In this multistudy analysis of 376 participants across 11 studies, we compared multivariate predictive models to investigate the spatial scale and location of evoked heat pain intensity representation. We compared models based on (a) a single most pain-predictive region or resting-state network; (b) pain-associated cortical–subcortical systems developed from prior literature (“multisystem models”); and (c) a model spanning the full brain. We estimated model accuracy using leave-one-study-out cross-validation (CV; 7 studies) and subsequently validated in 4 independent holdout studies. All spatial scales conveyed information about pain intensity, but distributed, multisystem models predicted pain 20% more accurately than any individual region or network and were more generalizable to multimodal pain (thermal, visceral, and mechanical) and specific to pain. Full brain models showed no predictive advantage over multisystem models. These findings show that multiple cortical and subcortical systems are needed to decode pain intensity, especially heat pain, and that representation of pain experience may not be circumscribed by any elementary region or canonical network. Finally, the learner generalization methods we employ provide a blueprint for evaluating the spatial scale of information in other domains.

Is pain represented by a single brain area or network, spanning multiple systems or distributed throughout the brain? fMRI brain decoding in a large multi-study dataset shows that multiple cortical and subcortical systems are needed to decode pain intensity; the approach is novel and can characterize scale of representation across diverse brain processes.
Strength-dependent perturbation of whole-brain model working in different regimes reveals the role of fluctuations in brain dynamics
Authors: Yonatan Sanz Perl, Anira Escrichs, Enzo Tagliazucchi, Morten L Kringelbach, Gustavo Deco
Journal: PLOS Computational Biology
Accepted: 2022-10-17T00:00:00Z


Despite decades of research, there is still a lack of understanding of the role and generating mechanisms of the ubiquitous fluctuations and oscillations found in recordings of brain dynamics. Here, we used whole-brain computational models capable of presenting different dynamical regimes to reproduce empirical data’s turbulence level. We showed that the model’s fluctuations regime fitted to turbulence more faithfully reproduces the empirical functional connectivity compared to oscillatory and noise regimes. By applying global and local strength-dependent perturbations and subsequently measuring the responsiveness of the model, we revealed each regime’s computational capacity demonstrating that brain dynamics is shifted towards fluctuations to provide much-needed flexibility. Importantly, fluctuation regime stimulation in a brain region within a given resting state network modulates that network, aligned with previous empirical and computational studies. Furthermore, this framework generates specific, testable empirical predictions for human stimulation studies using strength-dependent rather than constant perturbation. Overall, the whole-brain models fitted to the level of empirical turbulence together with functional connectivity unveil that the fluctuation regime best captures empirical data, and the strength-dependent perturbative framework demonstrates how this regime provides maximal flexibility to the human brain.
Author summary: How and why do complex, fluctuating, and oscillating dynamics characterise brain states? We combined a whole-brain model and strength-dependent perturbation frameworks to investigate the causal mechanistic explanation behind the human brain function. We demonstrated by fitting whole-brain models to the level of empirical turbulence together with functional connectivity that the fluctuation regime best captures empirical data. Furthermore, the strength-dependent perturbative approach allows us to assess the computational capabilities of different dynamical regimes. We showed that the fluctuations regime provides maximal flexibility to the human brain, a desirable property for brain dynamics to interact with the environment.
A multi-scale cortical wiring space links cellular architecture and functional dynamics in the human brain
Authors: Casey Paquola, Jakob Seidlitz, Oualid Benkarim, Jessica Royer, Petr Klimes, Richard A I Bethlehem, Sara Larivière, Reinder Vos de Wael, Raul Rodríguez-Cruces, Jeffery A Hall, Birgit Frauscher, Jonathan Smallwood, Boris C Bernhardt
Journal: PLOS Biology
Accepted: 2020-11-02T00:00:00Z


The vast net of fibres within and underneath the cortex is optimised to support the convergence of different levels of brain organisation. Here, we propose a novel coordinate system of the human cortex based on an advanced model of its connectivity. Our approach is inspired by seminal, but so far largely neglected models of cortico–cortical wiring established by postmortem anatomical studies and capitalises on cutting-edge in vivo neuroimaging and machine learning. The new model expands the currently prevailing diffusion magnetic resonance imaging (MRI) tractography approach by incorporation of additional features of cortical microstructure and cortico–cortical proximity. Studying several datasets and different parcellation schemes, we could show that our coordinate system robustly recapitulates established sensory-limbic and anterior–posterior dimensions of brain organisation. A series of validation experiments showed that the new wiring space reflects cortical microcircuit features (including pyramidal neuron depth and glial expression) and allowed for competitive simulations of functional connectivity and dynamics based on resting-state functional magnetic resonance imaging (rs-fMRI) and human intracranial electroencephalography (EEG) coherence. Our results advance our understanding of how cell-specific neurobiological gradients produce a hierarchical cortical wiring scheme that is concordant with increasing functional sophistication of human brain organisation. Our evaluations demonstrate the cortical wiring space bridges across scales of neural organisation and can be easily translated to single individuals.

This study proposes a novel coordinate system of the human cortex based on an expanded model of structural connectivity, and demonstrates that this system can visualize salient dimensions of brain organization, and reflects specific cytoarchitectural and cellular features. Functional neuroimaging and intracranial recordings show that the coordinate system can be used to make robust predictions of functional connectivity and brain dynamics.
Connectivity differences between Gulf War Illness (GWI) phenotypes during a test of attention
Authors: Tomas Clarke, Jessie D Jamieson, Patrick Malone, Rakib U Rayhan, Stuart Washington, John W VanMeter, James N Baraniuk
Journal: PLOS ONE
Accepted: 2019-11-26T00:00:00Z


One quarter of veterans returning from the 1990–1991 Persian Gulf War have developed Gulf War Illness (GWI) with chronic pain, fatigue, cognitive and gastrointestinal dysfunction. Exertion leads to characteristic, delayed onset exacerbations that are not relieved by sleep. We have modeled exertional exhaustion by comparing magnetic resonance images from before and after submaximal exercise. One third of the 27 GWI participants had brain stem atrophy and developed postural tachycardia after exercise (START: Stress Test Activated Reversible Tachycardia). The remainder activated basal ganglia and anterior insulae during a cognitive task (STOPP: Stress Test Originated Phantom Perception). Here, the role of attention in cognitive dysfunction was assessed by seed region correlations during a simple 0-back stimulus matching task (“see a letter, push a button”) performed before exercise. Analysis was analogous to resting state, but different from psychophysiological interactions (PPI). The patterns of correlations between nodes in task and default networks were significantly different for START (n = 9), STOPP (n = 18) and control (n = 8) subjects. Edges shared by the 3 groups may represent co-activation caused by the 0-back task. Controls had a task network of right dorsolateral and left ventrolateral prefrontal cortex, dorsal anterior cingulate cortex, posterior insulae and frontal eye fields (dorsal attention network). START had a large task module centered on the dorsal anterior cingulate cortex with direct links to basal ganglia, anterior insulae, and right dorsolateral prefrontal cortex nodes, and through dorsal attention network (intraparietal sulci and frontal eye fields) nodes to a default module. STOPP had 2 task submodules of basal ganglia–anterior insulae, and dorsolateral prefrontal executive control regions. Dorsal attention and posterior insulae nodes were embedded in the default module and were distant from the task networks. These three unique connectivity patterns during an attention task support the concept of Gulf War Disease with recognizable, objective patterns of cognitive dysfunction.
Learning-related congruent and incongruent changes of excitation and inhibition in distinct cortical areas
Authors: Vahid Esmaeili, Anastasiia Oryshchuk, Reza Asri, Keita Tamura, Georgios Foustoukos, Yanqi Liu, Romain Guiet, Sylvain Crochet, Carl C H Petersen
Journal: PLOS Biology
Accepted: 2022-05-10T00:00:00Z


Excitatory and inhibitory neurons in diverse cortical regions are likely to contribute differentially to the transformation of sensory information into goal-directed motor plans. Here, we investigate the relative changes across mouse sensorimotor cortex in the activity of putative excitatory and inhibitory neurons—categorized as regular spiking (RS) or fast spiking (FS) according to their action potential (AP) waveform—comparing before and after learning of a whisker detection task with delayed licking as perceptual report. Surprisingly, we found that the whisker-evoked activity of RS versus FS neurons changed in opposite directions after learning in primary and secondary whisker motor cortices, while it changed similarly in primary and secondary orofacial motor cortices. Our results suggest that changes in the balance of excitation and inhibition in local circuits concurrent with changes in the long-range synaptic inputs in distinct cortical regions might contribute to performance of delayed sensory-to-motor transformation.

A study of mouse sensorimotor cortex during a whisker detection task shows that learning of a goal-directed sensorimotor transformation is accompanied by differential changes in excitation and inhibition in distinct neocortical regions, helping to link sensory cortex and motor cortex for correct task performance.
Task-evoked activity quenches neural correlations and variability across cortical areas
Authors: Takuya Ito, Scott L Brincat, Markus Siegel, Ravi D Mill, Biyu J He, Earl K Miller, Horacio G Rotstein, Michael W Cole
Journal: PLOS Computational Biology
Accepted: 2020-05-27T00:00:00Z


Many large-scale functional connectivity studies have emphasized the importance of communication through increased inter-region correlations during task states. In contrast, local circuit studies have demonstrated that task states primarily reduce correlations among pairs of neurons, likely enhancing their information coding by suppressing shared spontaneous activity. Here we sought to adjudicate between these conflicting perspectives, assessing whether co-active brain regions during task states tend to increase or decrease their correlations. We found that variability and correlations primarily decrease across a variety of cortical regions in two highly distinct data sets: non-human primate spiking data and human functional magnetic resonance imaging data. Moreover, this observed variability and correlation reduction was accompanied by an overall increase in dimensionality (reflecting less information redundancy) during task states, suggesting that decreased correlations increased information coding capacity. We further found in both spiking and neural mass computational models that task-evoked activity increased the stability around a stable attractor, globally quenching neural variability and correlations. Together, our results provide an integrative mechanistic account that encompasses measures of large-scale neural activity, variability, and correlations during resting and task states.
Author summary: Statistical estimates of correlated neural activity and variability are widely used to characterize neural systems during different states. However, there is a conceptual gap between the use and interpretation of these measures between the human neuroimaging and non-human primate electrophysiology literature. For example, in the human neuroimaging literature, “functional connectivity” is often used to refer to correlated activity, while in the non-human primate electrophysiology literature, the equivalent term is “noise correlation”. In an effort to unify these two perspectives under a single theoretical framework, we provide empirical evidence from human functional magnetic resonance imaging and non-human primate mean-field spike rate data that functional connectivity and noise correlations reveal similar statistical patterns during task states. In short, we found that task states primarily quench neural variability and correlations in both data sets. To provide a theoretically rigorous account capable of explaining this phenomena across both data sets, we use mean-field dynamical systems modeling to demonstrate the deterministic relationship between task-evoked activity, neural variability and correlations. Together, we provide an integrative account, showing that task-evoked activity quenches neural variability and correlations in large-scale neural systems.
Cortical oscillations support sampling-based computations in spiking neural networks
Authors: Agnes Korcsak-Gorzo, Michael G Müller, Andreas Baumbach, Luziwei Leng, Oliver J Breitwieser, Sacha J van Albada, Walter Senn, Karlheinz Meier, Robert Legenstein, Mihai A Petrovici
Journal: PLOS Computational Biology
Accepted: 2021-12-14T00:00:00Z


Being permanently confronted with an uncertain world, brains have faced evolutionary pressure to represent this uncertainty in order to respond appropriately. Often, this requires visiting multiple interpretations of the available information or multiple solutions to an encountered problem. This gives rise to the so-called mixing problem: since all of these “valid” states represent powerful attractors, but between themselves can be very dissimilar, switching between such states can be difficult. We propose that cortical oscillations can be effectively used to overcome this challenge. By acting as an effective temperature, background spiking activity modulates exploration. Rhythmic changes induced by cortical oscillations can then be interpreted as a form of simulated tempering. We provide a rigorous mathematical discussion of this link and study some of its phenomenological implications in computer simulations. This identifies a new computational role of cortical oscillations and connects them to various phenomena in the brain, such as sampling-based probabilistic inference, memory replay, multisensory cue combination, and place cell flickering.
Author summary: Activity oscillations are a ubiquitous and well-studied phenomenon throughout the cortex. At the same time, mounting evidence suggests that brain networks perform sampling-based probabilistic inference through their dynamics. In this work, we present a theoretical and a computational analysis that establish a rigorous link between these two phenomena: background oscillations enhance sampling-based computations by helping networks of spiking neurons to quickly reach different high-probability network states, i.e., computational results.
Population heterogeneity in clinical cohorts affects the predictive accuracy of brain imaging
Authors: Oualid Benkarim, Casey Paquola, Bo-yong Park, Valeria Kebets, Seok-Jun Hong, Reinder Vos de Wael, Shaoshi Zhang, B T Thomas Yeo, Michael Eickenberg, Tian Ge, Jean-Baptiste Poline, Boris C Bernhardt, Danilo Bzdok
Journal: PLOS Biology
Accepted: 2022-04-11T00:00:00Z


Brain imaging research enjoys increasing adoption of supervised machine learning for single-participant disease classification. Yet, the success of these algorithms likely depends on population diversity, including demographic differences and other factors that may be outside of primary scientific interest. Here, we capitalize on propensity scores as a composite confound index to quantify diversity due to major sources of population variation. We delineate the impact of population heterogeneity on the predictive accuracy and pattern stability in 2 separate clinical cohorts: the Autism Brain Imaging Data Exchange (ABIDE, n = 297) and the Healthy Brain Network (HBN, n = 551). Across various analysis scenarios, our results uncover the extent to which cross-validated prediction performances are interlocked with diversity. The instability of extracted brain patterns attributable to diversity is located preferentially in regions part of the default mode network. Collectively, our findings highlight the limitations of prevailing deconfounding practices in mitigating the full consequences of population diversity.

Brain-imaging research enjoys increasing adoption of supervised machine learning for single-subject disease classification. This study explores the contribution of diversity-aware machine learning models to tracking, unpacking and understanding out-of-distribution generalization in large-scale neuroimaging datasets, and shows that population diversity is a key factor contributing to generalization performance.