PenguinMan/ARXIV huggingface dataset:
Combined real-time fMRI and real time fNIRS brain computer interface (BCI): Training of volitional wrist extension after stroke, a case series pilot study
Authors: Avi K Matarasso, Jake D Rieke, Keith White, M Minhal Yusufali, Janis J Daly
Journal: PLOS ONE
Accepted: 2021-04-01T00:00:00Z

Objective: Pilot testing of real time functional magnetic resonance imaging (rt-fMRI) and real time functional near infrared spectroscopy (rt-fNIRS) as brain computer interface (BCI) neural feedback systems combined with motor learning for motor recovery in chronic severely impaired stroke survivors. Approach: We enrolled a four-case series and administered three sequential rt-fMRI and ten rt-fNIRS neural feedback sessions interleaved with motor learning sessions. Measures were: Arm Motor Assessment Tool, functional domain (AMAT-F; 13 complex functional tasks), Fugl-Meyer arm coordination scale (FM); active wrist extension range of motion (ROM); volume of activation (fMRI); and fNIRS HbO concentration. Performance during neural feedback was assessed, in part, using percent successful brain modulations during rt-fNIRS. Main results: Pre-/post-treatment mean clinically significant improvement in AMAT-F (.49 ± 0.22) and FM (10.0 ± 3.3); active wrist ROM improvement ranged from 20° to 50°. Baseline to follow-up change in brain signal was as follows: fMRI volume of activation was reduced in almost all ROIs for three subjects, and for one subject there was an increase or no change; fNIRS HbO was within normal range, except for one subject who increased beyond normal at post-treatment. During rt-fNIRS neural feedback training, there was successful brain signal modulation (42%–78%). Significance: Severely impaired stroke survivors successfully engaged in spatially focused BCI systems, rt-fMRI and rt-fNIRS, to clinically significantly improve motor function. At the least, equivalency in motor recovery was demonstrated with prior long-duration motor learning studies (without neural feedback), indicating that no loss of motor improvement resulted from substituting neural feedback sessions for motor learning sessions. Given that the current neural feedback protocol did not prevent the motor improvements observed in other long duration studies, even in the presence of fewer sessions of motor learning in the current work, the results support further study of neural feedback and its potential for recovery of motor function in stroke survivors. In future work, expanding the sophistication of either or both rt-fMRI and rt-fNIRS could hold the potential for further reducing the number of hours of training needed and/or the degree of recovery.
Pupillometry in auditory multistability
Authors: Jan Grenzebach, Thomas G G Wegner, Wolfgang Einhäuser, Alexandra Bendixen
Journal: PLOS ONE
Accepted: 2021-05-15T00:00:00Z


In multistability, a constant stimulus induces alternating perceptual interpretations. For many forms of visual multistability, the transition from one interpretation to another (“perceptual switch”) is accompanied by a dilation of the pupil. Here we ask whether the same holds for auditory multistability, specifically auditory streaming. Two tones were played in alternation, yielding four distinct interpretations: the tones can be perceived as one integrated percept (single sound source), or as segregated with either tone or both tones in the foreground. We found that the pupil dilates significantly around the time a perceptual switch is reported (“multistable condition”). When participants instead responded to actual stimulus changes that closely mimicked the multistable perceptual experience (“replay condition”), the pupil dilated more around such responses than in multistability. This still held when data were corrected for the pupil response to the stimulus change as such. Hence, active responses to an exogeneous stimulus change trigger a stronger or temporally more confined pupil dilation than responses to an endogenous perceptual switch. In another condition, participants randomly pressed the buttons used for reporting multistability. In Study 1, this “random condition” failed to sufficiently mimic the temporal pattern of multistability. By adapting the instructions, in Study 2 we obtained a response pattern more similar to the multistable condition. In this case, the pupil dilated significantly around the random button presses. Albeit numerically smaller, this pupil response was not significantly different from the multistable condition. While there are several possible explanations–related, e.g., to the decision to respond–this underlines the difficulty to isolate a purely perceptual effect in multistability. Our data extend previous findings from visual to auditory multistability. They highlight methodological challenges in interpreting such data and suggest possible approaches to meet them, including a novel stimulus to simulate the experience of perceptual switches in auditory streaming.
Dynamic decorrelation as a unifying principle for explaining a broad range of brightness phenomena
Authors: Alejandro Lerer, Hans Supèr, Matthias S Keil
Journal: PLOS Computational Biology
Accepted: 2021-04-06T00:00:00Z


The visual system is highly sensitive to spatial context for encoding luminance patterns. Context sensitivity inspired the proposal of many neural mechanisms for explaining the perception of luminance (brightness). Here we propose a novel computational model for estimating the brightness of many visual illusions. We hypothesize that many aspects of brightness can be explained by a dynamic filtering process that reduces the redundancy in edge representations on the one hand, while non-redundant activity is enhanced on the other. The dynamic filter is learned for each input image and implements context sensitivity. Dynamic filtering is applied to the responses of (model) complex cells in order to build a gain control map. The gain control map then acts on simple cell responses before they are used to create a brightness map via activity propagation. Our approach is successful in predicting many challenging visual illusions, including contrast effects, assimilation, and reverse contrast with the same set of model parameters.
Author summary: We hardly notice that what we see is often different from the physical world “outside” of the brain. This means that the visual experience that the brain actively constructs may be different from the actual physical properties of objects in the world. In this work, we propose a hypothesis about how the visual system of the brain may construct a representation for achromatic images. Since this process is not unambiguous, sometimes we notice “errors” in our perception, which cause visual illusions. The challenge for theorists, therefore, is to propose computational principles that recreate a large number of visual illusions and to explain why they occur. Notably, our proposed mechanism explains a broader set of visual illusions than any previously published proposal. We achieved this by trying to suppress predictable information. For example, if an image contains repetitive structures, then these structures are predictable and will be suppressed. In this way, non-predictable structures stand out. Corresponding mechanisms act as early as in the retina (which enhances luminance changes but suppresses uniform regions of luminance), and our computational model suggests that such mechanisms also might be used at subsequent stages in the visual system, where representations of perceived luminance (=brightness) are created.
Retrieval (N400) and integration (P600) in expectation-based comprehension
Authors: Christoph Aurnhammer, Francesca Delogu, Miriam Schulz, Harm Brouwer, Matthew W Crocker
Journal: PLOS ONE
Accepted: 2021-08-31T00:00:00Z


Expectation-based theories of language processing, such as Surprisal theory, are supported by evidence of anticipation effects in both behavioural and neurophysiological measures. Online measures of language processing, however, are known to be influenced by factors such as lexical association that are distinct from—but often confounded with—expectancy. An open question therefore is whether a specific locus of expectancy related effects can be established in neural and behavioral processing correlates. We address this question in an event-related potential experiment and a self-paced reading experiment that independently cross expectancy and lexical association in a context manipulation design. We find that event-related potentials reveal that the N400 is sensitive to both expectancy and lexical association, while the P600 is modulated only by expectancy. Reading times, in turn, reveal effects of both association and expectancy in the first spillover region, followed by effects of expectancy alone in the second spillover region. These findings are consistent with the Retrieval-Integration account of language comprehension, according to which lexical retrieval (N400) is facilitated for words that are both expected and associated, whereas integration difficulty (P600) will be greater for unexpected words alone. Further, an exploratory analysis suggests that the P600 is not merely sensitive to expectancy violations, but rather, that there is a continuous relation. Taken together, these results suggest that the P600, like reading times, may reflect a meaning-centric notion of Surprisal in language comprehension.
Feature blindness: A challenge for understanding and modelling visual object recognition
Authors: Gaurav Malhotra, Marin Dujmović, Jeffrey S Bowers
Journal: PLOS Computational Biology
Accepted: 2022-03-19T00:00:00Z


Humans rely heavily on the shape of objects to recognise them. Recently, it has been argued that Convolutional Neural Networks (CNNs) can also show a shape-bias, provided their learning environment contains this bias. This has led to the proposal that CNNs provide good mechanistic models of shape-bias and, more generally, human visual processing. However, it is also possible that humans and CNNs show a shape-bias for very different reasons, namely, shape-bias in humans may be a consequence of architectural and cognitive constraints whereas CNNs show a shape-bias as a consequence of learning the statistics of the environment. We investigated this question by exploring shape-bias in humans and CNNs when they learn in a novel environment. We observed that, in this new environment, humans (i) focused on shape and overlooked many non-shape features, even when non-shape features were more diagnostic, (ii) learned based on only one out of multiple predictive features, and (iii) failed to learn when global features, such as shape, were absent. This behaviour contrasted with the predictions of a statistical inference model with no priors, showing the strong role that shape-bias plays in human feature selection. It also contrasted with CNNs that (i) preferred to categorise objects based on non-shape features, and (ii) increased reliance on these non-shape features as they became more predictive. This was the case even when the CNN was pre-trained to have a shape-bias and the convolutional backbone was frozen. These results suggest that shape-bias has a different source in humans and CNNs: while learning in CNNs is driven by the statistical properties of the environment, humans are highly constrained by their previous biases, which suggests that cognitive constraints play a key role in how humans learn to recognise novel objects.
Author summary: Any object consists of hundreds of visual features that can be used to recognise it. How do humans select which feature to use? Do we always choose features that are best at predicting the object? In a series of experiments using carefully designed stimuli, we find that humans frequently ignore many features that are clearly visible and highly predictive. This behaviour is statistically inefficient and we show that it contrasts with statistical inference models such as state-of-the-art neural networks. Unlike humans, these models learn to rely on the most predictive feature when trained on the same data. We argue that the reason underlying human behaviour may be a bias to look for features that are less hungry for cognitive resources and generalise better to novel instances. Models that incorporate cognitive constraints may not only allow us to better understand human vision but also help us develop machine learning models that are more robust to changes in incidental features of objects.
Gamma frequency sensory stimulation in mild probable Alzheimer’s dementia patients: Results of feasibility and pilot studies
Authors: Diane Chan, Ho-Jun Suk, Brennan L Jackson, Noah P Milman, Danielle Stark, Elizabeth B Klerman, Erin Kitchener, Vanesa S Fernandez Avalos, Gabrielle de Weck, Arit Banerjee, Sara D Beach, Joel Blanchard, Colton Stearns, Aaron D Boes, Brandt Uitermarkt, Phillip Gander, Matthew Howard III, Eliezer J Sternberg, Alfonso Nieto-Castanon, Sheeba Anteraper, Susan Whitfield-Gabrieli, Emery N Brown, Edward S Boyden, Bradford C Dickerson, Li-Huei Tsai
Journal: PLOS ONE
Accepted: 2022-10-11T00:00:00Z


Non-invasive Gamma ENtrainment Using Sensory stimulation (GENUS) at 40Hz reduces Alzheimer’s disease (AD) pathology such as amyloid and tau levels, prevents cerebral atrophy, and improves behavioral testing performance in mouse models of AD. Here, we report data from (1) a Phase 1 feasibility study (NCT04042922, ClinicalTrials.gov) in cognitively normal volunteers (n = 25), patients with mild AD dementia (n = 16), and patients with epilepsy who underwent intracranial electrode monitoring (n = 2) to assess safety and feasibility of a single brief GENUS session to induce entrainment and (2) a single-blinded, randomized, placebo-controlled Phase 2A pilot study (NCT04055376) in patients with mild probable AD dementia (n = 15) to assess safety, compliance, entrainment, and exploratory clinical outcomes after chronic daily 40Hz sensory stimulation for 3 months. Our Phase 1 study showed that 40Hz GENUS was safe and effectively induced entrainment in both cortical regions and other cortical and subcortical structures such as the hippocampus, amygdala, insula, and gyrus rectus. Our Phase 2A study demonstrated that chronic daily 40Hz light and sound GENUS was well-tolerated and that compliance was equally high in both the control and active groups, with participants equally inaccurate in guessing their group assignments prior to unblinding. Electroencephalography recordings show that our 40Hz GENUS device safely and effectively induced 40Hz entrainment in participants with mild AD dementia. After 3 months of daily stimulation, the group receiving 40Hz stimulation showed (i) lesser ventricular dilation and hippocampal atrophy, (ii) increased functional connectivity in the default mode network as well as with the medial visual network, (iii) better performance on the face-name association delayed recall test, and (iv) improved measures of daily activity rhythmicity compared to the control group. These results support further evaluation of GENUS in a pivotal clinical trial to evaluate its potential as a novel disease-modifying therapeutic for patients with AD.
Two stages of bandwidth scaling drives efficient neural coding of natural sounds
Authors: Fengrong He, Ian H Stevenson, Monty A Escabí
Journal: PLOS Computational Biology
Accepted: 2023-01-09T00:00:00Z


Theories of efficient coding propose that the auditory system is optimized for the statistical structure of natural sounds, yet the transformations underlying optimal acoustic representations are not well understood. Using a database of natural sounds including human speech and a physiologically-inspired auditory model, we explore the consequences of peripheral (cochlear) and mid-level (auditory midbrain) filter tuning transformations on the representation of natural sound spectra and modulation statistics. Whereas Fourier-based sound decompositions have constant time-frequency resolution at all frequencies, cochlear and auditory midbrain filters bandwidths increase proportional to the filter center frequency. This form of bandwidth scaling produces a systematic decrease in spectral resolution and increase in temporal resolution with increasing frequency. Here we demonstrate that cochlear bandwidth scaling produces a frequency-dependent gain that counteracts the tendency of natural sound power to decrease with frequency, resulting in a whitened output representation. Similarly, bandwidth scaling in mid-level auditory filters further enhances the representation of natural sounds by producing a whitened modulation power spectrum (MPS) with higher modulation entropy than both the cochlear outputs and the conventional Fourier MPS. These findings suggest that the tuning characteristics of the peripheral and mid-level auditory system together produce a whitened output representation in three dimensions (frequency, temporal and spectral modulation) that reduces redundancies and allows for a more efficient use of neural resources. This hierarchical multi-stage tuning strategy is thus likely optimized to extract available information and may underlies perceptual sensitivity to natural sounds.
Author summary: Theory suggests that the auditory system evolved to optimally encode salient structure in natural sounds—maximizing perceptual capabilities while minimizing metabolic demands. Here, using a multi-stage model of the auditory system and a collection of environmental sounds, including vocalizations such as speech, we demonstrate how auditory responses may be optimized for equalizing the power distribution of natural sounds at two levels. This processing strategy may improve the allocation of resources throughout the auditory pathway, while ensuring that a broad range of auditory features can be detected and perceived. Such a multi-stage strategy for processing natural sounds likely contributes to human perceptual capabilities and adopting such a code could enhance the performance of auditory prosthetics and machine systems for sound recognition.
I wanna hold your hand: Handholding is preferred over gentle stroking for emotion regulation
Authors: Haran Sened, Chen Levin, Manar Shehab, Naomi Eisenberger, Simone Shamay-Tsoory
Journal: PLOS ONE
Accepted: 2023-03-24T00:00:00Z


Social touch is an important form of interpersonal emotion regulation. In recent years, the emotion regulation effects of two types of touch have been studied extensively: handholding and stroking (specifically of skin with C-tactile afferents on the forearm, i.e. C-touch). While some studies compare their effectiveness, with mixed results, no study to date has examined which type of touch is subjectively preferred. Given the potential bidirectional communication provided by handholding, we hypothesized that to regulate intense emotions, participants would prefer handholding. In four pre-registered online studies (total N = 287), participants rated handholding and stroking, presented in short videos, as emotion regulation methods. Study 1 examined touch reception preference in hypothetical situations. Study 2 replicated Study 1 while also examining touch provision preferences. Study 3 examined touch reception preferences of participants with blood/injection phobia in hypothetical injection situations. Study 4 examined types of touch participants who have recently given birth recalled receiving during childbirth and their hypothetical preferences. In all studies, participants preferred handholding over stroking; participants who have recently given birth reported receiving handholding more than stroking. This was especially evident in Studies 1–3 in emotionally intense situations. These results demonstrate that handholding is preferred over stroking as a form of emotion regulation, especially in intense situations, and support the importance of two-way sensory communication for emotion regulation via touch. We discuss the results and possible additional mechanisms, including top-down processing and cultural priming.
Bioinspired figure-ground discrimination via visual motion smoothing
Authors: Zhihua Wu, Aike Guo
Journal: PLOS Computational Biology
Accepted: 2023-04-04T00:00:00Z


Flies detect and track moving targets among visual clutter, and this process mainly relies on visual motion. Visual motion is analyzed or computed with the pathway from the retina to T4/T5 cells. The computation of local directional motion was formulated as an elementary movement detector (EMD) model more than half a century ago. Solving target detection or figure-ground discrimination problems can be equivalent to extracting boundaries between a target and the background based on the motion discontinuities in the output of a retinotopic array of EMDs. Individual EMDs cannot measure true velocities, however, due to their sensitivity to pattern properties such as luminance contrast and spatial frequency content. It remains unclear how local directional motion signals are further integrated to enable figure-ground discrimination. Here, we present a computational model inspired by fly motion vision. Simulations suggest that the heavily fluctuating output of an EMD array is naturally surmounted by a lobula network, which is hypothesized to be downstream of the local motion detectors and have parallel pathways with distinct directional selectivity. The lobula network carries out a spatiotemporal smoothing operation for visual motion, especially across time, enabling the segmentation of moving figures from the background. The model qualitatively reproduces experimental observations in the visually evoked response characteristics of one type of lobula columnar (LC) cell. The model is further shown to be robust to natural scene variability. Our results suggest that the lobula is involved in local motion-based target detection.
Author summary: Flies are adept at target tracking during high-speed aerial maneuvers, although their brains are tiny. Along the pathway from the retina to the deep optic lobe, local directional motion at different positions within the visual field is first detected, based on which optic flow the fly experiences is then encoded. In contrast to extensive research on optic flow processing in the fly visual system, much less is known regarding how target detection is performed, even under the simple condition of a tethered fly. In this study, we built a fly inspired model to investigate how a textured figure is discriminated from a similarly textured background by relying on relative motion cues alone. We found that the secret involved in a network downstream of local motion detectors extracting the figure is to naturally exploit the temporal coherence of the moving figure. Without a response integration in time to a sequence of temporally ordered input frames, the downstream network fails to extract the figure, even if it has made full use of spatial coherence between adjacent pixels in individual frames. The modeling is helpful for understanding where and how target detection is performed in the visual system of flies. The result could also be inspiring for motion detection in machine vision systems that rely only on a single camera.