PenguinMan/ARXIV huggingface dataset:
Learning and forgetting using reinforced Bayesian change detection
Authors: Vincent Moens, Alexandre Zénon
Journal: PLOS Computational Biology
Accepted: 2018-12-09T00:00:00Z


Agents living in volatile environments must be able to detect changes in contingencies while refraining to adapt to unexpected events that are caused by noise. In Reinforcement Learning (RL) frameworks, this requires learning rates that adapt to past reliability of the model. The observation that behavioural flexibility in animals tends to decrease following prolonged training in stable environment provides experimental evidence for such adaptive learning rates. However, in classical RL models, learning rate is either fixed or scheduled and can thus not adapt dynamically to environmental changes. Here, we propose a new Bayesian learning model, using variational inference, that achieves adaptive change detection by the use of Stabilized Forgetting, updating its current belief based on a mixture of fixed, initial priors and previous posterior beliefs. The weight given to these two sources is optimized alongside the other parameters, allowing the model to adapt dynamically to changes in environmental volatility and to unexpected observations. This approach is used to implement the “critic” of an actor-critic RL model, while the actor samples the resulting value distributions to choose which action to undertake. We show that our model can emulate different adaptation strategies to contingency changes, depending on its prior assumptions of environmental stability, and that model parameters can be fit to real data with high accuracy. The model also exhibits trade-offs between flexibility and computational costs that mirror those observed in real data. Overall, the proposed method provides a general framework to study learning flexibility and decision making in RL contexts.
Author summary: In stable contexts, animals and humans exhibit automatic behaviour that allows them to make fast decisions. However, these automatic processes exhibit a lack of flexibility when environmental contingencies change. In the present paper, we propose a model of behavioural automatization that is based on adaptive forgetting and that emulates these properties. The model builds an estimate of the stability of the environment and uses this estimate to adjust its learning rate and the balance between exploration and exploitation policies. The model performs Bayesian inference on latent variables that represent relevant environmental properties, such as reward functions, optimal policies or environment stability. From there, the model makes decisions in order to maximize long-term rewards, with a noise proportional to environmental uncertainty. This rich model encompasses many aspects of Reinforcement Learning (RL), such as Temporal Difference RL and counterfactual learning, and accounts for the reduced computational cost of automatic behaviour. Using simulations, we show that this model leads to interesting predictions about the efficiency with which subjects adapt to sudden change of contingencies after prolonged training.
Integrative genomic analysis of early neurogenesis reveals a temporal genetic program for differentiation and specification of preplate and Cajal-Retzius neurons
Authors: Jia Li, Lei Sun, Xue-Liang Peng, Xiao-Ming Yu, Shao-Jun Qi, Zhi John Lu, Jing-Dong J Han, Qin Shen
Journal: PLOS Genetics
Accepted: 2021-01-12T00:00:00Z


Neurogenesis in the developing neocortex begins with the generation of the preplate, which consists of early-born neurons including Cajal-Retzius (CR) cells and subplate neurons. Here, utilizing the Ebf2-EGFP transgenic mouse in which EGFP initially labels the preplate neurons then persists in CR cells, we reveal the dynamic transcriptome profiles of early neurogenesis and CR cell differentiation. Genome-wide RNA-seq and ChIP-seq analyses at multiple early neurogenic stages have revealed the temporal gene expression dynamics of early neurogenesis and distinct histone modification patterns in early differentiating neurons. We have identified a new set of coding genes and lncRNAs involved in early neuronal differentiation and validated with functional assays in vitro and in vivo. In addition, at E15.5 when Ebf2-EGFP+ cells are mostly CR neurons, single-cell sequencing analysis of purified Ebf2-EGFP+ cells uncovers molecular heterogeneities in CR neurons, but without apparent clustering of cells with distinct regional origins. Along a pseudotemporal trajectory these cells are classified into three different developing states, revealing genetic cascades from early generic neuronal differentiation to late fate specification during the establishment of CR neuron identity and function. Our findings shed light on the molecular mechanisms governing the early differentiation steps during cortical development, especially CR neuron differentiation.
Author summary: Neural stem cells and progenitor cells in the embryonic brain give rise to neurons following a precise temporal order after initial expansion. Early-born neurons including Cajal-Retzius (CR) cells and subplate neurons form the preplate in the developing cerebral cortex, then CR neurons occupy the layer 1, playing an important role in cortical histogenesis. The molecular mechanisms governing the early neuronal differentiation processes remain to be explored. Here, by genome-wide approaches including bulk RNA-seq, single-cell RNA-seq and ChIP-seq, we comprehensively characterized the temporal dynamic gene expression profile and epigenetic status at different stages during early cortical development and uncovered molecularly heterogeneous subpopulations within the CR cells. We revealed CR neuron signatures and cell type-specific histone modification patterns along early neuron specification. Using in vitro and in vivo assays, we identified novel lncRNAs as potential functional regulators in preplate differentiation and CR neuron identity establishment. Our study provides a comprehensive analysis of the genetic and epigenetic programs during neuronal differentiation and would help bring new insights into the early cortical neurogenesis process, particularly the differentiation of CR neurons.
Attention controls multisensory perception via two distinct mechanisms at different levels of the cortical hierarchy
Authors: Ambra Ferrari, Uta Noppeney
Journal: PLOS Biology
Accepted: 2021-11-01T00:00:00Z


To form a percept of the multisensory world, the brain needs to integrate signals from common sources weighted by their reliabilities and segregate those from independent sources. Previously, we have shown that anterior parietal cortices combine sensory signals into representations that take into account the signals’ causal structure (i.e., common versus independent sources) and their sensory reliabilities as predicted by Bayesian causal inference. The current study asks to what extent and how attentional mechanisms can actively control how sensory signals are combined for perceptual inference. In a pre- and postcueing paradigm, we presented observers with audiovisual signals at variable spatial disparities. Observers were precued to attend to auditory or visual modalities prior to stimulus presentation and postcued to report their perceived auditory or visual location. Combining psychophysics, functional magnetic resonance imaging (fMRI), and Bayesian modelling, we demonstrate that the brain moulds multisensory inference via two distinct mechanisms. Prestimulus attention to vision enhances the reliability and influence of visual inputs on spatial representations in visual and posterior parietal cortices. Poststimulus report determines how parietal cortices flexibly combine sensory estimates into spatial representations consistent with Bayesian causal inference. Our results show that distinct neural mechanisms control how signals are combined for perceptual inference at different levels of the cortical hierarchy.

A combination of psychophysics, computational modelling and fMRI reveals novel insights into how the brain controls the binding of information across the senses, such as the voice and lip movements of a speaker.
Kinesin-3 mediated axonal delivery of presynaptic neurexin stabilizes dendritic spines and postsynaptic components
Authors: Devyn Oliver, Shankar Ramachandran, Alison Philbrook, Christopher M Lambert, Ken C Q Nguyen, David H Hall, Michael M Francis
Journal: PLOS Genetics
Accepted: 2022-01-03T00:00:00Z


The functional properties of neural circuits are defined by the patterns of synaptic connections between their partnering neurons, but the mechanisms that stabilize circuit connectivity are poorly understood. We systemically examined this question at synapses onto newly characterized dendritic spines of C. elegans GABAergic motor neurons. We show that the presynaptic adhesion protein neurexin/NRX-1 is required for stabilization of postsynaptic structure. We find that early postsynaptic developmental events proceed without a strict requirement for synaptic activity and are not disrupted by deletion of neurexin/nrx-1. However, in the absence of presynaptic NRX-1, dendritic spines and receptor clusters become destabilized and collapse prior to adulthood. We demonstrate that NRX-1 delivery to presynaptic terminals is dependent on kinesin-3/UNC-104 and show that ongoing UNC-104 function is required for postsynaptic maintenance in mature animals. By defining the dynamics and temporal order of synapse formation and maintenance events in vivo, we describe a mechanism for stabilizing mature circuit connectivity through neurexin-based adhesion.
Author summary: The nervous system is composed of networks of interconnecting cells called neurons. Within these networks, individual neurons establish connections or synapses with partnering neurons. Key connections within these networks must be stabilized and preserved throughout the lifetime of an animal in order for the nervous system to function properly. Alterations in synapse connectivity and stability are linked with numerous neurodevelopmental and neurodegenerative diseases. Therefore, it is crucial to gain a complete understanding of the steps in assembly of a developing synapse, and of the mechanisms involved in synapse maturation and maintenance. Using microscopy and live imaging approaches at newly characterized spiny synapses in a genetic model organism, the nematode Caenorhabdtis elegans, we describe the sequence of events by which developing synapses are constructed. In addition, we show that the cell adhesion molecule, neurexin, is required for stabilization of these synapses into adulthood. In the absence of neurexin, neuronal connections begin to form but then disappear. A specific motor protein, called kinesin-3, is required to deliver neurexin to synapses in order to stabilize specific connections between neurons. An improved understanding of the sequence of events in synapse formation and how neurexin is delivered to developing synapses for stabilizing connections may help to advance our understanding of disorders and diseases that arise from disruptions of these processes.
Are you confident enough to act? Individual differences in action control are associated with post-decisional metacognitive bias
Authors: Wojciech Zajkowski, Maksymilian Bielecki, Magdalena Marszał-Wiśniewska
Journal: PLOS ONE
Accepted: 2022-05-03T00:00:00Z


The art of making good choices and being consistent in executing them is essential for having a successful and fulfilling life. Individual differences in action control are believed to have a crucial impact on how we make choices and whether we put them in action. Action-oriented people are more decisive, flexible and likely to implement their intentions in the face of adversity. In contrast, state-oriented people often struggle to commit to their choices and end up second-guessing themselves. Here, we employ a model-based computational approach to study the underlying cognitive differences between action and state-oriented people in simple binary-choice decision tasks. In Experiment 1 we show that there is little-to-no evidence that the two groups differ in terms of decision-related parameters and strong evidence for differences in metacognitive bias. Action-oriented people exhibit greater confidence in the correctness of their choices as well as slightly elevated judgement sensitivity, although no differences in performance are present. In Experiment 2 we replicate this effect and show that the confidence gap generalizes to value-based decisions, widens as a function of difficulty and is independent of deliberation interval. Furthermore, allowing more time for confidence deliberation indicated that state-oriented people focus more strongly on external features of choice. We propose that a positive confidence bias, coupled with appropriate metacognitive sensitivity, might be crucial for the successful realization of intentions in many real-life situations. More generally, our study provides an example of how modelling latent cognitive processes can bring meaningful insight into the study of individual differences.
Structure learning enhances concept formation in synthetic Active Inference agents
Authors: Victorita Neacsu, M Berk Mirza, Rick A Adams, Karl J Friston
Journal: PLOS ONE
Accepted: 2022-10-24T00:00:00Z


Humans display astonishing skill in learning about the environment in which they operate. They assimilate a rich set of affordances and interrelations among different elements in particular contexts, and form flexible abstractions (i.e., concepts) that can be generalised and leveraged with ease. To capture these abilities, we present a deep hierarchical Active Inference model of goal-directed behaviour, and the accompanying belief update schemes implied by maximising model evidence. Using simulations, we elucidate the potential mechanisms that underlie and influence concept learning in a spatial foraging task. We show that the representations formed–as a result of foraging–reflect environmental structure in a way that is enhanced and nuanced by Bayesian model reduction, a special case of structure learning that typifies learning in the absence of new evidence. Synthetic agents learn associations and form concepts about environmental context and configuration as a result of inferential, parametric learning, and structure learning processes–three processes that can produce a diversity of beliefs and belief structures. Furthermore, the ensuing representations reflect symmetries for environments with identical configurations.
Differences in the functional brain architecture of sustained attention and working memory in youth and adults
Authors: Omid Kardan, Andrew J Stier, Carlos Cardenas-Iniguez, Kathryn E Schertz, Julia C Pruin, Yuting Deng, Taylor Chamberlain, Wesley J Meredith, Xihan Zhang, Jillian E Bowman, Tanvi Lakhtakia, Lucy Tindel, Emily W Avery, Qi Lin, Kwangsun Yoo, Marvin M Chun, Marc G Berman, Monica D Rosenberg
Journal: PLOS Biology
Accepted: 2022-11-30T00:00:00Z


Sustained attention (SA) and working memory (WM) are critical processes, but the brain networks supporting these abilities in development are unknown. We characterized the functional brain architecture of SA and WM in 9- to 11-year-old children and adults. First, we found that adult network predictors of SA generalized to predict individual differences and fluctuations in SA in youth. A WM model predicted WM performance both across and within children—and captured individual differences in later recognition memory—but underperformed in youth relative to adults. We next characterized functional connections differentially related to SA and WM in youth compared to adults. Results revealed 2 network configurations: a dominant architecture predicting performance in both age groups and a secondary architecture, more prominent for WM than SA, predicting performance in each age group differently. Thus, functional connectivity (FC) predicts SA and WM in youth, with networks predicting WM performance differing more between youths and adults than those predicting SA.

Which brain networks support sustained attention (SA) and working memory (WM) during development? This study shows that adult functional brain network models predict individual differences and within-person changes in pre-adolescents’ SA and WM performance.
The effect of music on body sway when standing in a moving virtual environment
Authors: Shaquitta Dent, Kelley Burger, Skyler Stevens, Benjamin D Smith, Jefferson W Streepey
Journal: PLOS ONE
Accepted: 2021-09-15T00:00:00Z


Movement of the visual environment presented through virtual reality (VR) has been shown to invoke postural adjustments measured by increased body sway. The effect of auditory information on body sway seems to be dependent on context with sounds such as white noise, tones, and music being used to amplify or suppress sway. This study aims to show that music manipulated to match VR motion further increases body sway. Twenty-eight subjects stood on a force plate and experienced combinations of 3 visual conditions (VR translation in the AP direction at 0.1 Hz, no translation, and eyes closed) and 4 music conditions (Mozart’s Jupiter Symphony modified to scale volume at 0.1 Hz and 0.25 Hz, unmodified music, and no music) Body sway was assessed by measuring center of pressure (COP) velocities and RMS. Cross-coherence between the body sway and the 0.1 Hz and 0.25 Hz stimuli was also determined. VR translations at 0.1 Hz matched with 0.1Hz shifts in music volume did not lead to more body sway than observed in the no music and unmodified music conditions. Researchers and clinicians may consider manipulating sound to enhance VR induced body sway, but findings from this study would not suggest using volume to do so.
Covert spatial selection in primate basal ganglia
Authors: Fabrice Arcizet, Richard J Krauzlis
Journal: PLOS Biology
Accepted: 2018-10-16T00:00:00Z


The basal ganglia are important for action selection. They are also implicated in perceptual and cognitive functions that seem far removed from motor control. Here, we tested whether the role of the basal ganglia in selection extends to nonmotor aspects of behavior by recording neuronal activity in the caudate nucleus while animals performed a covert spatial attention task. We found that caudate neurons strongly select the spatial location of the relevant stimulus throughout the task even in the absence of any overt action. This spatially selective activity was dependent on task and visual conditions and could be dissociated from goal-directed actions. Caudate activity was also sufficient to correctly identify every epoch in the covert attention task. These results provide a novel perspective on mechanisms of attention by demonstrating that the basal ganglia are involved in spatial selection and tracking of behavioral states even in the absence of overt orienting movements.
Author summary: The ability to respond selectively to sensory inputs is a crucial brain function, one that is implicated in variety of high-level brain disorders. The basal ganglia are a set of evolutionarily ancient structures best known for their role in controlling motor actions and more recently implicated in nonmotor cognitive functions. Here we show for the first time, to our knowledge, that neuronal activity in the caudate nucleus, one of the major input structures of the basal ganglia, is modulated during a covert selective attention task. Two monkeys were trained to detect a subtle change in a visual motion stimulus at a cued location while ignoring a similar stimulus at a second, uncued location. Monkeys were not allowed to look directly at the stimuli, but they monitored them covertly and released a joystick to report their choices. We found that caudate neurons strongly selected the spatial location of the relevant stimulus throughout the task even in the absence of any overt movement. This spatially selective activity was dependent on task and visual conditions and could be dissociated from goal-directed actions. These results advance our understanding of the basal ganglia and provide a novel perspective on the brain circuits for selective attention.
Modeling driver cells in developing neuronal networks
Authors: Stefano Luccioli, David Angulo-Garcia, Rosa Cossart, Arnaud Malvache, Laura Módol, Vitor Hugo Sousa, Paolo Bonifazi, Alessandro Torcini
Journal: PLOS Computational Biology
Accepted: 2018-10-07T00:00:00Z


Spontaneous emergence of synchronized population activity is a characteristic feature of developing brain circuits. Recent experiments in the developing neo-cortex showed the existence of driver cells able to impact the synchronization dynamics when single-handedly stimulated. We have developed a spiking network model capable to reproduce the experimental results, thus identifying two classes of driver cells: functional hubs and low functionally connected (LC) neurons. The functional hubs arranged in a clique orchestrated the synchronization build-up, while the LC drivers were lately or not at all recruited in the synchronization process. Notwithstanding, they were able to alter the network state when stimulated by modifying the temporal activation of the functional clique or even its composition. LC drivers can lead either to higher population synchrony or even to the arrest of population dynamics, upon stimulation. Noticeably, some LC driver can display both effects depending on the received stimulus. We show that in the model the presence of inhibitory neurons together with the assumption that younger cells are more excitable and less connected is crucial for the emergence of LC drivers. These results provide a further understanding of the structural-functional mechanisms underlying synchronized firings in developing circuits possibly related to the coordinated activity of cell assemblies in the adult brain.
Author summary: There is timely interest on the impact of peculiar neurons (driver cells) and of small neuronal sub-networks (cliques) on operational brain dynamics. We first provide experimental data concerning the effect of stimulated driver cells on the bursting activity observable in the developing entorhinal cortex. Secondly, we develop a network model able to fully reproduce the experimental observations. Analogously to the experiments two types of driver cells can be identified: functional hubs and low functionally connected (LC) cells. We explain the role of hub neurons, arranged in a clique, for the orchestration of the bursting activity in control conditions. Furthermore, we report a new mechanism which can explain why and how LC drivers emerge in the structural-functional organization of the entorhinal cortex.