PenguinMan/ARXIV huggingface dataset:
Decoupled neural network training with re-computation and weight prediction
Authors: Jiawei Peng, Yicheng Xu, Zhiping Lin, Zhenyu Weng, Zishuo Yang, Huiping Zhuang
Journal: PLOS ONE
Accepted: 2022-10-04T00:00:00Z


To break the three lockings during backpropagation (BP) process for neural network training, multiple decoupled learning methods have been investigated recently. These methods either lead to significant drop in accuracy performance or suffer from dramatic increase in memory usage. In this paper, a new form of decoupled learning, named decoupled neural network training scheme with re-computation and weight prediction (DTRP) is proposed. In DTRP, a re-computation scheme is adopted to solve the memory explosion problem, and a weight prediction scheme is proposed to deal with the weight delay caused by re-computation. Additionally, a batch compensation scheme is developed, allowing the proposed DTRP to run faster. Theoretical analysis shows that DTRP is guaranteed to converge to crical points under certain conditions. Experiments are conducted by training various convolutional neural networks on several classification datasets, showing comparable or better results than the state-of-the-art methods and BP. These experiments also reveal that adopting the proposed method, the memory explosion problem is effectively solved, and a significant acceleration is achieved.
Input correlations impede suppression of chaos and learning in balanced firing-rate networks
Authors: Rainer Engelken, Alessandro Ingrosso, Ramin Khajeh, Sven Goedeke, L F Abbott
Journal: PLOS Computational Biology
Accepted: 2022-09-20T00:00:00Z


Neural circuits exhibit complex activity patterns, both spontaneously and evoked by external stimuli. Information encoding and learning in neural circuits depend on how well time-varying stimuli can control spontaneous network activity. We show that in firing-rate networks in the balanced state, external control of recurrent dynamics, i.e., the suppression of internally-generated chaotic variability, strongly depends on correlations in the input. A distinctive feature of balanced networks is that, because common external input is dynamically canceled by recurrent feedback, it is far more difficult to suppress chaos with common input into each neuron than through independent input. To study this phenomenon, we develop a non-stationary dynamic mean-field theory for driven networks. The theory explains how the activity statistics and the largest Lyapunov exponent depend on the frequency and amplitude of the input, recurrent coupling strength, and network size, for both common and independent input. We further show that uncorrelated inputs facilitate learning in balanced networks.
Author summary: Information in the brain is processed by a deeply-layered structure of local recurrent neural circuits. Recurrent neural networks often exhibit spontaneous irregular activity patterns that arise generically through the disordered interactions between neurons. Understanding under which conditions one circuit can control the activity patterns in another circuit and suppress spontaneous, chaotic fluctuations is crucial to unravel information flow and learning input-output tasks.
Multi-region electricity demand prediction with ensemble deep neural networks
Authors: Muhammad Irfan, Ahmad Shaf, Tariq Ali, Mariam Zafar, Saifur Rahman, Salim Nasar Faraj Mursal, Faisal AlThobiani, Majid A. Almas, H M Attar, Nagi Abdussamiee
Journal: PLOS ONE
Accepted: 2023-04-24T00:00:00Z


Electricity consumption prediction plays a vital role in intelligent energy management systems, and it is essential for electricity power supply companies to have accurate short and long-term energy predictions. In this study, a deep-ensembled neural network was used to anticipate hourly power utilization, providing a clear and effective approach for predicting power consumption. The dataset comprises of 13 files, each representing a different region, and ranges from 2004 to 2018, with two columns for the date, time, year and energy expenditure. The data was normalized using minmax scalar, and a deep ensembled (long short-term memory and recurrent neural network) model was used for energy consumption prediction. This proposed model effectively trains long-term dependencies in sequence order and has been assessed using several statistical metrics, including root mean squared error (RMSE), relative root mean squared error (rRMSE), mean absolute bias error (MABE), coefficient of determination (R2), mean bias error (MBE), and mean absolute percentage error (MAPE). Results show that the proposed model performs exceptionally well compared to existing models, indicating its effectiveness in accurately predicting energy consumption.
Predicting vasospasm risk using first presentation aneurysmal subarachnoid hemorrhage volume: A semi-automated CT image segmentation analysis using ITK-SNAP
Authors: James S Street, Anand S Pandit, Ahmed K Toma
Journal: PLOS ONE
Accepted: 2023-05-17T00:00:00Z

Purpose: Cerebral vasospasm following aneurysmal subarachnoid hemorrhage (aSAH) is a significant complication associated with poor neurological outcomes. We present a novel, semi-automated pipeline, implemented in the open-source medical imaging analysis software ITK-SNAP, to segment subarachnoid blood volume from initial CT head (CTH) scans and use this to predict future radiological vasospasm. Methods: 42 patients were admitted between February 2020 and December 2021 to our tertiary neurosciences center, and whose initial referral CTH scan was used for this retrospective cohort study. Blood load was segmented using a semi-automated random forest classifier and active contour evolution implemented in ITK-SNAP. Clinical data were extracted from electronic healthcare records in order to fit models aimed at predicting radiological vasospasm risk. Results: Semi-automated segmentations demonstrated excellent agreement with manual, expert-derived volumes (mean Dice coefficient = 0.92). Total normalized blood volume, extracted from CTH images at first presentation, was significantly associated with greater odds of later radiological vasospasm, increasing by approximately 7% for each additional cm3 of blood (OR = 1.069, 95% CI: 1.021–1.120; p < .005). Greater blood volume was also significantly associated with vasospasm of a higher Lindegaard ratio, of longer duration, and a greater number of discrete episodes. Total blood volume predicted radiological vasospasm with a greater accuracy as compared to the modified Fisher scale (AUC = 0.86 vs 0.70), and was of independent predictive value. Conclusion: Semi-automated methods provide a plausible pipeline for the segmentation of blood from CT head images in aSAH, and total blood volume is a robust, extendable predictor of radiological vasospasm, outperforming the modified Fisher scale. Greater subarachnoid blood volume significantly increases the odds of subsequent vasospasm, its time course and its severity.
Discovery of hierarchical representations for efficient planning
Authors: Momchil S Tomov, Samyukta Yagati, Agni Kumar, Wanqian Yang, Samuel J Gershman
Journal: PLOS Computational Biology
Accepted: 2019-12-10T00:00:00Z


We propose that humans spontaneously organize environments into clusters of states that support hierarchical planning, enabling them to tackle challenging problems by breaking them down into sub-problems at various levels of abstraction. People constantly rely on such hierarchical presentations to accomplish tasks big and small—from planning one’s day, to organizing a wedding, to getting a PhD—often succeeding on the very first attempt. We formalize a Bayesian model of hierarchy discovery that explains how humans discover such useful abstractions. Building on principles developed in structure learning and robotics, the model predicts that hierarchy discovery should be sensitive to the topological structure, reward distribution, and distribution of tasks in the environment. In five simulations, we show that the model accounts for previously reported effects of environment structure on planning behavior, such as detection of bottleneck states and transitions. We then test the novel predictions of the model in eight behavioral experiments, demonstrating how the distribution of tasks and rewards can influence planning behavior via the discovered hierarchy, sometimes facilitating and sometimes hindering performance. We find evidence that the hierarchy discovery process unfolds incrementally across trials. Finally, we propose how hierarchy discovery and hierarchical planning might be implemented in the brain. Together, these findings present an important advance in our understanding of how the brain might use Bayesian inference to discover and exploit the hidden hierarchical structure of the environment.
Author summary: Human planning is hierarchical. Whether planning something simple like cooking dinner or something complex like a trip abroad, we usually begin with a rough mental sketch of the goals we want to achieve (“go to Spain, then go back home”). This sketch is then progressively refined into a detailed sequence of sub-goals (“book flight”, “pack luggage”), sub-sub-goals, and so on, down to the actual sequence of bodily movements that is much more complicated than the original plan. Efficient planning therefore requires knowledge of the abstract high-level concepts that constitute the essence of hierarchical plans. Yet how humans learn such abstractions remains a mystery. In this study, we show that humans spontaneously form such high-level concepts in a way that allows them to plan efficiently given the tasks, rewards, and structure of their environment. We also show that this behavior is consistent with a formal model of the underlying computations, thus grounding these findings in established computational principles and relating them to previous studies of hierarchical planning. We believe our results pave the way for future studies to investigate the neural algorithms that support this essential cognitive ability.
Dynamic patterns of correlated activity in the prefrontal cortex encode information about social behavior
Authors: Nicholas A Frost, Anna Haggart, Vikaas S Sohal
Journal: PLOS Biology
Accepted: 2021-04-14T00:00:00Z


New technologies make it possible to measure activity from many neurons simultaneously. One approach is to analyze simultaneously recorded neurons individually, then group together neurons which increase their activity during similar behaviors into an “ensemble.” However, this notion of an ensemble ignores the ability of neurons to act collectively and encode and transmit information in ways that are not reflected by their individual activity levels. We used microendoscopic GCaMP imaging to measure prefrontal activity while mice were either alone or engaged in social interaction. We developed an approach that combines a neural network classifier and surrogate (shuffled) datasets to characterize how neurons synergistically transmit information about social behavior. Notably, unlike optimal linear classifiers, a neural network classifier with a single linear hidden layer can discriminate network states which differ solely in patterns of coactivity, and not in the activity levels of individual neurons. Using this approach, we found that surrogate datasets which preserve behaviorally specific patterns of coactivity (correlations) outperform those which preserve behaviorally driven changes in activity levels but not correlated activity. Thus, social behavior elicits increases in correlated activity that are not explained simply by the activity levels of the underlying neurons, and prefrontal neurons act collectively to transmit information about socialization via these correlations. Notably, this ability of correlated activity to enhance the information transmitted by neuronal ensembles is diminished in mice lacking the autism-associated gene Shank3. These results show that synergy is an important concept for the coding of social behavior which can be disrupted in disease states, reveal a specific mechanism underlying this synergy (social behavior increases correlated activity within specific ensembles), and outline methods for studying how neurons within an ensemble can work together to encode information.

Behaviorally-specific patterns of correlated activity between prefrontal neurons normally enhance the information that neuronal ensembles transmit about social behavior. This study shows that in a mouse model of autism, individual neurons continue to encode social information, but this additional information carried by patterns of correlated activity is lost.
A cluster-randomized controlled trial of the effectiveness of the JUMP Math program of math instruction for improving elementary math achievement
Authors: Tracy Solomon, Annie Dupuis, Arland O’Hara, Min-Na Hockenberry, Jenny Lam, Geraldine Goco, Bruce Ferguson, Rosemary Tannock
Journal: PLOS ONE
Accepted: 2019-09-12T00:00:00Z


Students in many western countries struggle to achieve acceptable standards in numeracy despite its recognition as an important 21st century skill. As commercial math programs remain a staple of classroom instruction, investigations of their effectiveness are essential to inform decision-making regarding how to invest limited resources while maximizing student gains. We conducted a cluster randomized-controlled trial of the effectiveness of JUMP Math, a distinctive math program whose central tenets are empirically supported, for improving elementary math achievement (clinical trial.gov no. NCT02456181). The study involved 554 grade 2 (primary) and 592 grade 5 (junior) students and 193 teachers in 41 schools, in an urban-rural Canadian school board. Schools were randomly assigned to use either JUMP Math or their business-as-usual, problem-based approach to math instruction. We tracked student progress in math achievement on standardized and curriculum-based measures of computation and problem solving, for 2 consecutive school years. Junior students taught with JUMP Math made significantly greater progress in computation than their non-JUMP peers but the groups did not differ significantly in problem solving. Effects took hold relatively quickly, replicating the results from an earlier pilot study. Primary students in the non-JUMP group made significantly greater gains in problem solving and computation in year 1. But those taught with JUMP Math made significantly greater gains in problem solving and the groups did not differ in computation, in year 2. The positive effects of JUMP Math are noteworthy given that the JUMP Math teachers were likely still adjusting to the new program. That these positive findings were obtained in an effectiveness study (i.e. in real-world conditions), suggests that JUMP Math may be a valuable evidence-based addition to the teacher’s toolbox. Given the importance of numeracy for 21st century functioning, identifying and implementing effective math instruction programs could have far-reaching, positive implications.
Associations between resting state functional brain connectivity and childhood anhedonia: A reproduction and replication study
Authors: Yi Zhou, Narun Pat, Michael C Neale
Journal: PLOS ONE
Accepted: 2023-02-28T00:00:00Z

Background: Previously, a study using a sample of the Adolescent Brain Cognitive Development (ABCD)® study from the earlier 1.0 release found differences in several resting state functional MRI (rsfMRI) brain connectivity measures associated with children reporting anhedonia. Here, we aim to reproduce, replicate, and extend the previous findings using data from the later ABCD study 4.0 release, which includes a significantly larger sample. Methods: To reproduce and replicate the previous authors’ findings, we analyzed data from the ABCD 1.0 release (n = 2437), from an independent subsample from the newer ABCD 4.0 release (excluding individuals from the 1.0 release) (n = 6456), and from the full ABCD 4.0 release sample (n = 8866). Additionally, we assessed whether using a multiple linear regression approach could improve replicability by controlling for the effects of comorbid psychiatric conditions and sociodemographic covariates. Results: While the previously reported associations were reproducible, effect sizes for most rsfMRI measures were drastically reduced in replication analyses (including for both t-tests and multiple linear regressions) using the ABCD 4.0 (excluding 1.0) sample. However, 2 new rsfMRI measures (the Auditory vs. Right Putamen and the Retrosplenial-Temporal vs. Right-Thalamus-Proper measures) exhibited replicable associations with anhedonia and stable, albeit small, effect sizes across the ABCD samples, even after accounting for sociodemographic covariates and comorbid psychiatric conditions using a multiple linear regression approach. Conclusion: The most statistically significant associations between anhedonia and rsfMRI connectivity measures found in the ABCD 1.0 sample tended to be non-replicable and inflated. Contrastingly, replicable associations exhibited smaller effects with less statistical significance in the ABCD 1.0 sample. Multiple linear regressions helped assess the specificity of these findings and control the effects of confounding covariates.
Preference and strategy in proposer’s prosocial giving in the ultimatum game
Authors: Misato Inaba, Yumi Inoue, Satoshi Akutsu, Nobuyuki Takahashi, Toshio Yamagishi
Journal: PLOS ONE
Accepted: 2018-02-19T00:00:00Z


The accumulation of findings that most responders in the ultimatum game reject unfair offers provides evidence that humans are driven by social preferences such as preferences for fairness and prosociality. On the other hand, if and how the proposer’s behavior is affected by social preferences remains unelucidated. We addressed this question for the first time by manipulating the knowledge that the proposer had about the responder’s belief concerning the intentionality of the proposer. In a new game called the “ultimatum game with ambiguous intentions of the proposer (UGAMB),” we made the intentionality of the proposer ambiguous to the recipient. We expected and found that the proposer would make more unfair offers in this new game than in the standard ultimatum game. This expectation can be derived from either the preference-based model or the strategy model of the proposer’s giving decision. The additional finding that more unfair giving in the UGAMB was not mediated by the proposer’s expectation that the recipient would be more willing to accept unfair offers provided support for the preference-based model. Using a psychological measure of cognitive control, the preference-based model received additional support through a conceptual replication of the previous finding that cognitive control of intuitive drive for prosociality in the dictator game, rather than mind reading in the ultimatum game, is responsible for the difference in giving between the two games.
A shared synapse architecture for efficient FPGA implementation of autoencoders
Authors: Akihiro Suzuki, Takashi Morie, Hakaru Tamukoh
Journal: PLOS ONE
Accepted: 2018-02-25T00:00:00Z


This paper proposes a shared synapse architecture for autoencoders (AEs), and implements an AE with the proposed architecture as a digital circuit on a field-programmable gate array (FPGA). In the proposed architecture, the values of the synapse weights are shared between the synapses of an input and a hidden layer, and between the synapses of a hidden and an output layer. This architecture utilizes less of the limited resources of an FPGA than an architecture which does not share the synapse weights, and reduces the amount of synapse modules used by half. For the proposed circuit to be implemented into various types of AEs, it utilizes three kinds of parameters; one to change the number of layers’ units, one to change the bit width of an internal value, and a learning rate. By altering a network configuration using these parameters, the proposed architecture can be used to construct a stacked AE. The proposed circuits are logically synthesized, and the number of their resources is determined. Our experimental results show that single and stacked AE circuits utilizing the proposed shared synapse architecture operate as regular AEs and as regular stacked AEs. The scalability of the proposed circuit and the relationship between the bit widths and the learning results are also determined. The clock cycles of the proposed circuits are formulated, and this formula is used to estimate the theoretical performance of the circuit when the circuit is used to construct arbitrary networks.
Detection of epileptic seizure based on entropy analysis of short-term EEG
Authors: Peng Li, Chandan Karmakar, John Yearwood, Svetha Venkatesh, Marimuthu Palaniswami, Changchun Liu
Journal: PLOS ONE
Accepted: 2018-02-19T00:00:00Z


Entropy measures that assess signals’ complexity have drawn increasing attention recently in biomedical field, as they have shown the ability of capturing unique features that are intrinsic and physiologically meaningful. In this study, we applied entropy analysis to electroencephalogram (EEG) data to examine its performance in epilepsy detection based on short-term EEG, aiming at establishing a short-term analysis protocol with optimal seizure detection performance. Two classification problems were considered, i.e., 1) classifying interictal and ictal EEGs (epileptic group) from normal EEGs; and 2) classifying ictal from interictal EEGs. For each problem, we explored two protocols to analyze the entropy of EEG: i) using a single analytical window with different window lengths, and ii) using an average of multiple windows for each window length. Two entropy methods—fuzzy entropy (FuzzyEn) and distribution entropy (DistEn)–were used that have valid outputs for any given data lengths. We performed feature selection and trained classifiers based on a cross-validation process. The results show that performance of FuzzyEn and DistEn may complement each other and the best performance can be achieved by combining: 1) FuzzyEn of one 5-s window and the averaged DistEn of five 1-s windows for classifying normal from epileptic group (accuracy: 0.93, sensitivity: 0.91, specificity: 0.96); and 2) the averaged FuzzyEn of five 1-s windows and DistEn of one 5-s window for classifying ictal from interictal EEGs (accuracy: 0.91, sensitivity: 0.93, specificity: 0.90). Further studies are warranted to examine whether this proposed short-term analysis procedure can help track the epileptic activities in real time and provide prompt feedback for clinical practices.