dataesr/scientific-paragraphs-categorization huggingface dataset:
The model loss is the weighted sum of a reconstruction loss and the Kullback-Leibner (KL) loss. The total loss function can be dynamically changed in the training process for KL-term annealing  43  ; as standard practice when working with discrete data such as language. Our two final models were trained without and with the KL-term annealing (VAE_v1 and VAE_v2, respectively). The models were compiled using the Adam optimizer. Pretraining and training. Our training dataset of ~5,000 AMP sequences is not sufficient for learning what makes a protein sequence distinct from a random string of amino acid characters and what makes a protein sequence an AMP. In such cases, pretraining with a much bigger generic dataset is needed to enhance the model performance. We pretrained the generator models with ~1.
Acknowledgments This work was supported by a  European Molecular Biology Organization (EMBO)  long-term postdoctoral fellowship to A. P. ( ALTG 165-2020 ), the  Max Planck Society, German Research Council DFG ;  AB 792/1-1 (423428279 ) to F. A. ,  German Research Council   TRR81 : ' Chromatin Changes in Differentiation and Malignancies '  TRR81/3 ,  Z04  ( 109546710 ) to V. T. T. and O. V. , the  Bundesministerium für Bildung und Forschung (Federal Ministry of Education and Research, PermedCOPD -FKZ   01EK2203A ;  ERACoSysMed2 -SysMed-COPD -FKZ 031L0140 ), the  Deutsche Forschungsgemeinschaft  ( SFB/TR-84 TP C01 ) to B. S. , the  Hessisches Ministerium für Wissenschaft und Kunst  ( LOEWE Diffusible Signals LOEWE-Schwerpunkt Diffusible Signals ) to A. L. J and B. S. S. LS. and G. H. acknowledge financial support by the  Clusterprojekt ENABLE  funded by the  Hessian Ministry for Science and the Arts , and the  Collaborative Research Center 1507  funded by the  Deutsche Forschungsgemeinschaft (DFG, German Research Foundation ), and thank the  Max Planck Computing and Data Facility (MPCDF)  for computational resources.
M. K. acknowledges funding support from  INRAe's MICA department, Université Paris-Saclay ,  Ilede-France (IdF) region's DIM-RFSI, ANR DREAMY  ( ANR-21-CE48-003 , and  ANR  iCFree grant ( ANR-20-BiopNSE ). We thank  R. Weiss ,  P. Wichmann ,  A. M. Küffner ,  S. Scholz ,  R. Inckemann ,  C. Diehl ,  A. Yazdizadeh Kharrazi ,  J. Zarzycki , and  H. He  for technical and experimental support, as well as fruitful discussion. Figures were created with Biorender. com.
We provide a code for the new protocol that can be used October 2023, 10(10) ENEURO. 0145-23. 2023 1-13 Open Source Tools and Methods and adapted to common types of operant boxes. In addition, we provide a set of scripts that allow automatic extraction and analysis of numerous parameters recorded during each session. We demonstrate that the analysis of multiple performance parameters is necessary for detailed insight into the behavior of animals during the task. We validate our paradigm in an experiment using the valproate model of autism as a model of cognitive inflexibility. We show that the valproate mice show superior performance at specific stages of the task, paradoxically because of their propensity to more stereotypic behavior.
Once the complex sequence is well trained, it can be repeatedly modified at different steps, to implement the flexibility component of the task. In addition, the task structure allows collection of data and analysis of numerous behavioral parameters. The present protocol can be realized with commonly available operant boxes equipped only with two nonretractable levers and a feeder. The protocol code is programmed in a proprietary but common programming language that is used with a common type of operant boxes. The code is open and can be easily modified to create new variations of the task. To shorten the tedious analysis of multiple task parameters, we also provide a set of scripts in Python for automatized processing of the data and for user-friendly manual checking of lengthy task logs if needed.
Timed sequence task: data analysis The boxes were operated by the MedPC-V software that was also used to record all events that were defined in the task code. The resulting task log shows all successive events in a row (one row per session) as decimal numbers: the number before decimal point indicates the time of the respective event that elapsed from the last recorded event. The number after the decimal point indicates the specific event using a respective code as defined in the task. The task events recorded in the log included individual lever presses, both correct and incorrect, and presses during the TO interval, and reward delivery after the completion of correct sequences. Main summary parameters for each session (e. g. , total number of presses) were recorded in a text file generated for each session.
For more detailed analysis, we generated a comprehensive task log containing all sessions of the study and analyzed various performance parameters using the log and the analysis scripts created for that purpose. In addition to the individual analysis scripts, we also created a script allowing an easy and quick searching in the extensive task logs and visualization of defined time points during the session. The complete task log containing data from this study can be found at https://doi. org/10. 5281/zenodo. 7875357. A summary of all task codes and parameters we used is shown in Tables 2-Tables 4. The codes and the analysis scripts are available at https://doi. org/10. 5281/zenodo. 7881059.
Statistical analysis We used 22 mice in total, 11 mice per group (control vs VPA). Outliers were defined as values outside the range The depicted parameters follow the interface of the Med-PC V software (Med Associates). They may slightly differ when adapted to a different type of software. (average 6 2 Â SD), and they were excluded from the analysis. The identified outliers are indicated in figure legends together with respective sample sizes. Statistical analysis was performed with GraphPad Prism version 8 (GraphPad Software). We used two t tests and one-way ANOVAs for evaluating the effect of one variable and twoway ANOVA or repeated-measures (RM) two-way ANOVA for evaluating the effects of two variables. Sidak's test was used for post hoc comparison. For the analysis of the cumulative data, we used a nonparametric Mann-Whitney test to compare individual data points between individual groups of animals.
The depicted parameters follow the interface of the Med-PC V software (Med Associates). they may slightly differ when adapted to a different type of software. The depicted parameters follow the interface of the Med-PC V software (MedAssociates). they may slightly differ when adapted to a different type of software. 4b), both control and VPA mice significantly decreased the number of incorrect presses and presses emitted during the time-out (TO presses; Fig. 4c, d ). Initially, VPA mice performed fewer TO presses compared with controls, but the difference was less prominent after training (Fig. 4e, f ). To understand the activity of mice during the session, we quantified individual types of pressing sequences, correct and incorrect, performed in the first and last LLRR baseline session. Unlike the first session, in the last session mice performed significantly more correct sequences than any other type of sequence (Fig.
4d ). There are currently several alternatives that allow a detailed testing of motor or operant learning and flexibility. Among those, the touchscreen-based systems offer a great number of predesigned paradigms and allows creating additional, custom-made tasks  (Horner et al. , 2013; Mar et al. , 2013; Heath et al. , 2016; Janickova et al. , 2021). However, a significant cost of such systems may be prohibitive for some users. Standard operant boxes, on the other hand, are easily accessible to most laboratories. We show here that even the most basic operant box equipped with two fixed levers and one feeder can be used to perform a relatively complex task and collect detailed behavioral data. We provide a computer code for the task itself and also a set of scripts for data analysis, allowing easy and fast processing of large datasets.
Implementation We have implemented our method using the DOLFINX finite element solver of the FEniCS Project  [12]. Each parametric subproblem is submitted to a batch job queue. A distinct MPI communicator is used for each job. We use a standard first-order Lagrange finite element method and the resulting linear system is solved using the conjugate gradient method preconditioned using BoomerAMG from HYPRE  [56]  via the interface in PETSc  [16]. To compute the Bank-Weiser error estimator for each subproblem we use the methodology outlined in  [36]  and implemented in the FEniCSx-EE package  [35]. For every subproblem the computed solution and error estimate is written to disk in HDF5 format. A final step, running on a single MPI communicator, reads the solutions and error estimates for all subproblems, computes the quadrature sums using axpy operations, defines the marked set of cells to be refined using the Dörfler algorithm  [53] , and finally refines the mesh using the Plaza-Carey algorithm  [77].
Acknowledgments R. B. would like to acknowledge the support of the  ASSIST  research project of the  University of Luxembourg. This publication has been prepared in the framework of the  DRIVEN TWINNING  project funded by the  European Union 's  Horizon 2020 Research and Innovation programme  under Grant Agreement No. 811099. F. C. 's work is partially supported by the  I-Site BFC  project  NAANoD and the EIPHI Graduate School  (contract  ANR-17-EURE-0002 ).
Contrarily to backpropagation, Equilibrium Propagation uses the same computations in the forward and backward phases, which is another highly desirable feature for neuromorphic systems as it greatly simplifies the circuits. Equilibrium propagation is, however, originally a rate-based algorithm. Here, we design a spiking, hardware-friendly version of Equilibrium Propagation, called EqSpike, compatible with current neuromorphic technologies achieving online learning  (26) (27) (28) (29) (30) (31) (32). EqSpike is local in space and time: contrarily to backpropagation, neither error gradients nor activations need to be stored in external memories, and synapses can be directly updated through neural events. We simulate a fully connected network based on this architecture on the MNIST handwritten digits database.
For training, synaptic values are updated by probing the neuron states after  (21)  or during  (33)  the nudging phase through a learning rule that has been shown theoretically and numerically to match the updates of Back-Propagation Through Time, the state-of-the-art algorithm for such recurrent neural networks  (24). It has been shown recently that Equilibrium Propagation also reaches accuracy within 1% of BPTT with convolutional architectures on the CIFAR-10 dataset  (25). The original version of Equilibrium Propagation uses a rate-based formulation where dynamical neurons evolve smoothly in time. For a network of leaky integrate and fire neurons described by an Hopfield-like energy function 𝐸(𝑢) = 1 2 ⁄ ∑ 𝑢 𝑖 , where 𝑢 are the membrane potentials of neurons and 𝜌 their activation function, the Equilibrium Propagation learning rule is: ∆𝑊 𝑖𝑗 ~ (𝜌 𝑖 𝜌 𝑗 ) 𝑛 -(𝜌 𝑖 𝜌 𝑗 ) 𝑓 , where the product 𝜌 𝑖 𝜌 𝑗 is measured at equilibrium, at the end of the nudge phase and the free phase  (21).
Full network simulations: recognition rate on handwritten digits database We now evaluate the performance of EqSpike on the MNIST handwritten digits classification task, using a fully connected network with one hidden layer (see Supplementary Information for details). The obtained train (orange) and test (blue) accuracies are shown Fig. 2  as a function of the number of training epochs, with the deviation over six runs in shadow color. Table  1  compares the results to BPTT and the version of Equilibrium Propagation closest to our implementation, called Continual Eq-Prop  (33) , trained with a batch size of one. The test accuracy of EqSpike matches closely the accuracy of stochastic gradient descent through BPTT on the same network architecture, given the error margin. With a hidden layer of 300 neurons, EqSpikes reaches a test accuracy of 97.
59%. Fully-connected spiking neural networks trained on MNIST without conversion from a non-spiking neural network typically achieve recognition rates in the 96-98% range  (13, (39) (40) (41) (42). EqSpike, with its local, two-factors online learning rule therefore reaches accuracies on MNIST comparable to those of the latest models investigated for training spiking neural networks on hardware platforms. We have chosen the MNIST dataset as a benchmark because it is a standard dataset for the neuromorphic community interested in training spiking neural networks online, due to the long simulation times. As EqSpike achieves results on MNIST equivalent to the baseline given by Equilibrium Propagation it has the potential, like Equilibrium Propagation, to adapt to convolutional architectures and perform with good accuracy on more complex image benchmarks  (25) , with the additional advantage of being compatible with current neuromorphic technologies.
Inference speed and energy As EqSpike is derived from a rate-based algorithm, it is interesting for neuromorphic applications to quantify the number of spikes needed to achieve inference, and the time needed to reach high accuracy. Operation with fewer spikes is more desirable, as it reduces both execution time and energy consumption. Fig. 3  shows the inference results as a function of the execution time multiplied by the maximum frequency of neurons (𝑡 × 𝑓 𝑚𝑎𝑥 ). The orange line is the mean accuracy result over the whole test dataset, obtained by computing the spike rate of output neurons (as done for Fig. 2  and Table  1 ), through averaging in a time window 𝑇 𝑎𝑣𝑒𝑟𝑎𝑔𝑒 of 100 simulation time steps (𝑇 𝑎𝑣𝑒𝑟𝑎𝑔𝑒 = 50/𝑓 𝑚𝑎𝑥 ) and considering that the neuron with highest frequency encodes the output.
This method computes the rate accurately at the expense of having to wait the time 𝑇 𝑎𝑣𝑒𝑟𝑎𝑔𝑒 and letting output neurons spike multiple times. Spiking neural networks also offer the possibility to accelerate the computation and reduce the energy consumption by determining the output class from the first output neuron to spike. The blue line in Fig. 3  is the accuracy as a function of time, averaged over all images in the test dataset, obtained by considering that the first output neuron to spike encodes the output. The red, vertical dotted line in Fig. 3  indicates the average time of the first spike at the output over all presented images, corresponding to t  3. 5/𝑓 𝑚𝑎𝑥. At t  10/𝑓 𝑚𝑎𝑥 , the accuracy of single-spike inference (blue curve) reaches 95. 11% ± 0. 78%, within 1. 4% of the precise rate computation (orange curve).
Synaptic operations (SynOps) are defined as the total number of spikes transiting through synapses of the network. Frenkel et al show that a SynOp on a neuromorphic chip requires as little as 10 pJ  (30). The total number of synaptic operations needed for inference depends on the targeted recognition precision and, therefore, on the duration of inference (Fig. 3 ). For EqSpike, the recognition rate saturates at t 10/𝑓 𝑚𝑎𝑥. The corresponding measured number of SynOps is about 150,000 in average. This is much less than expected if all neurons spiked. This is also a bit less but comparable to the number of SynOps needed at inference for Event-Driven Random BackPropagation  (13). Considering 10 pJ/SynOps, each EqSpike inference could potentially consume 1. 5 µJ. This means that testing the 10,000 images of the whole MNIST dataset could be achieved with a neuromorphic chip while consuming only 15 mJ, in other words, three orders of magnitude less than with a GPU  (45).
Training speed and energy Training with EqSpike requires performing the free phase and then the nudging phase, during which synaptic weights are updated. A way to speed-up the training, and reduce the total number of SynOps, is to perform the nudging phase only on poorly classified examples, and skip the updates when the accuracy is satisfactory. We apply this strategy inspired from (32) using the criteria that the nudging phase is performed only when the difference between the target rate and the actual rate (𝜌 𝑘 ̂-𝜌 𝑘 ) at the ouput is above 1%. Fig. 4a  shows the number of presented examples per epoch as a function of epoch number. In the last 20 epochs only approximately 15% of the training dataset still require a nudging phase. For MNIST, we thus perform the free phase on all the dataset (3×10 6 images for the 50 epochs), and the nudging phase on 489,000 images.
Given the durations of each phase, we can estimate the training time to Ttraining  2. 74×10 8 /𝑓 𝑚𝑎𝑥. For electronic neurons with a firing rate of 100 kHz  (26) , this leads to Ttraining  45 min, and for electronic neurons with a firing rate of 10 MHz  (43)  to Ttraining  30s. As our networks feature a fully parallel nature, these training times would be the same for much wider networks, and increased by a factor of about eight only with four hidden layers  (25). As EqSpike is derived from a rate-based approach, it is interesting to compare the actual spiking rates of neurons in the network during training to their maximum frequency 𝑓 𝑚𝑎𝑥. For neuromorphic applications, low overall rates are indeed desirable. Fig  4b  shows the average number of spikes emitted by each neuron for an image presentation in the training dataset, as a function of the epoch.
5b. According to the Eq-Prop learning rule, in a network with unidirectional synapses, 𝑑𝑊 𝑖𝑗 𝑑𝑡 ∝ 𝜌̇𝑗 𝜌 𝑖 = 𝜌̇𝑝 𝑜𝑠𝑡 𝜌 𝑝𝑟𝑒 , a positive weight update should be applied. Due to the acceleration of the post-neuron, there are less post-neuron spikes before the pre-neuron spike than after. Therefore, tpost -tpre is positive in average, yielding a positive weight update through STDP. We have investigated if STDP-like weight updates did emerge during learning in our simulations. For this purpose, we monitored weight variations in synapses that connect input neurons and the hidden layer neurons. These synapses are unidirectional as input neurons are clamped to the input (see Fig. 1 ): their frequency does not vary. We used 100 images during the first epoch for the MNIST dataset. The curve in Fig. 5c  shows the average weight updates in the first layer as a function of the time difference between post-synaptic neuron spikes and the average time of pre-synaptic neuron spikes in a window of 200 time steps before the post-synaptic neuron spike.
In the first original research paper, Liang et al. introduce two primary contributions. The first is a multi-source joint-domain adaptation network proposal that addresses the challenge of crossdomain generalization in electroencephalography (EEG) emotion recognition. The second involves extensive cross-subject and crosssession transfer experiments on a publicly available emotion EEG dataset, validating the effectiveness of the proposed method. The second original research paper, by Ran et al. , presents three key contributions. First, the authors propose a simplified style transfer mapping method based on the instance selection (SSTM-IS) algorithm, which enhances the accuracy and speed of crosssubject emotion recognition by selecting informative instances and simplifying the update strategy of hyperparameters in style transfer mapping.
Second, the proposed algorithm is validated on both public and self-collected datasets, demonstrating higher accuracy in a shorter computing time for real-time emotion recognition applications. Third, the authors design and implement a real-time emotion recognition system that integrates EEG signal acquisition, data processing, emotion recognition, and result visualization. In the third original research paper, Li et al. introduce a novel model, STGATE, which combines a Transformer Learning Block (TLB) and a Spatial-Temporal Graph Attention (STGAT) mechanism. The TLB leverages 2D convolutional layers and a transformer encoder to extract time-frequency information, while the STGAT incorporates spatial and temporal attention mechanisms to capture connections between brain regions and temporal information.
The project has contributed to charting human brain development across the lifespan (0-100 years) in an international teamwork led by the Lifespan Brain Chart Consortium (LBCC)  18. To expand available resources for investigating population diversity  19  while recognizing and addressing the issues of sampling bias, and inclusion barriers within developmental population neuroscience  20  , we describe and share the brain-mind datasets of devCCNP here. We offer a comprehensive outline of the devCCNP protocol, along with recommendations to ensure that devCCNP can be scaled up to facilitate access to more diverse populations in the future. We provide all the anonymized raw data adhering to Brain Imaging Data Structure (BIDS) standards  21. In summary, this dataset comprises ample tasks addressing neurodevelopmental milestones of both primary and higher-order cognitive functions.
The dataset holds the potential to deepen our understanding of brain development in various dimensions, and augments assessments of cultural diversity among the existing datasets using accelerated longitudinal designs (ALD) (see Table  1  from the cohort profile on CCNP  12  for a nonexhaustive list of normative developmental samples obtained by ALD). In addition, we hope that the devCCNP will provide a resource to explore potential regional differences due to multisite sampling, and their impacts on brain development.
Methods overall design. The first stage of devCCNP aimed to establish an ALD cohort. The cohort consisted of 480 participants with typical development, who were evenly divided into age-specific groups. Each age cohort contain 20 boys and 20 girls (Fig. 1a ). We conducted data collection in two regions in China with distinct geographic and socioeconomic profiles, the Beibei District of Chongqing (devCCNP-CKG Sample) and the Chaoyang District of Beijing(devCCNP-PEK Sample), to capture a more representative sample of the Chinese population and its diverse characteristics. The devCCNP-CKG Sample was collected from March 2013 to January 2017, and the devCCNP-PEK Sample was collected from September 2017 to December 2022. Participants underwent assessment three times in total, referred to as three "waves" of visits.
To account for season effects, there was a 15-month time gap between each wave (Fig. 1b ). A repeated protocol was applied, which was adjusted based on the participants' age. The total time for each assessment was approximately 10-12 hours, including preparation time and short breaks. The time duration of each visit is listed in Supplementary Table  S1. Recruitment strategy. The devCCNP project focused on enrolling typically developing school-age Chinese children and adolescents. The CKG Sample was included one primary school and one junior high school in Chongqing. The participants were recruited through face-to-face communications between parents, schools, and CCNP program staff. In the case of the PEK Sample, recruitment took place in Beijing, where community-based Fig. 1  Experimental design and sample composition.
Dots indicate the specific age of each wave's data collection, while lines indicate the actual intervals between two waves. (e,g) Numbers of participants enrolled (Wave 1) in each age group are calculated according to sex. (f) The actual intervals in the CKG Sample better adhere to the original design; the largest interval is 19 months. (h) In the PEK Sample, intervals have been commonly extended from 16 to 50 months. recruitment was initially accomplished through various science popularization activities and online advertisements. We provided a series of activities for the families to experience educational neuroscience, including lectures on the brain, neuroimaging, cognitive neuroscience, and facility tours to experience MRI mock scanning, to make them interested and familiar with the entire procedure.