dataesr/scientific-paragraphs-categorization huggingface dataset:
We discuss in § 7 secure RPSs, required in adversarial networks such as blockchains, and avenues to secure GPS. Target system. We chose to simulate the Ethereum system as it is a large-scale and well-studied system with a diverse set of available datasets documenting its behaviors. Since we cannot always find accurate approximations of all the parameters required to properly simulate the Ethereum network, we sometimes use those of Bitcoin. As described in § 6. 1, we use a scenario where 10 updates (blocks) are created (by miners). We evaluate two configurations, Uniform gossip and GPS, using the following described datasets to obtain realistic results. Topology. We use 22,982 nodes in total, including 328 miners, as the most recent study on Ethereum shows  [10, Tab. 2]. While the number of miners may seem low, note that miners may form mining pools.
MS data processing MS data were processed using default parameters of the MaxQuant software (v1. 5. 2. 8, 1. 6. 2. 10 or v1. 6. 14. 0; a final analysis with the version 1. 6. 14. 0 was performed to ensure reproducibility of the coverage across the analysis)  [16]. Extracted peak lists were submitted to database search using the Andromeda search engine  [17]  to query a target-decoy  [21]  database of Homo sapiens (96,817 entries, downloaded on the 11th of December 2019) and 285 commonly observed contaminants. In database search, full tryptic specificity was required and up to two missed cleavages were allowed. Carbamidomethylation of cysteine was set as fixed modification, whereas protein N-terminal acetylation, and oxidation of methionine were set as variable modifications. In addition,  1 3  phosphorylation of serine, threonine, and tyrosine were set as variable modifications for phosphoproteome analysis.
Initial precursor mass tolerance was set to 4. 5 ppm, and 20 ppm at the fragment ion level. Peptide, protein and modification site identifications were filtered at a false discovery rate (FDR) of 0. 01. The iBAQ (Intensity Based Absolute Quantification) and LFQ (Label-Free Quantification) algorithms were enabled, as was the "match between runs" option  [44, 67]. The MS proteomics data have been deposited to the Pro-teomeXchange Consortium (http:// prote omece ntral. prote omexc hange. org) via the PRIDE partner repository  [55]  and are available with the dataset identifiers PXD041543 (hiPSC-derivewd MNs) and PXD042617 (post mortem samples). Detailed information can also be found in Supplementary Table  2 , online resource.
2% TWEEN, incubated with HRP-conjugated secondary Ab for 1 h, and again washed 3 times. Chemiluminescent signal was detected using the ECL detection kit (ThermoFisher Scientific, 32,106) and a MicroChemi 4. 2 device (DNR Bio Imaging System). For quantification, Gel-analyzer Software 2010a was used. The values of the proteins of interest were normalized against the loading control, β-actin.
Microscopy Fluorescence microscopy was performed with a Thunder imaging system (Leica) equipped with a DFC9000 sCMOS camera, an HC PL Fluotar 20X air (N. A. 0. 4) objective and using the LasX software (Leica). Confocal microscopy was performed with a laser-scanning microscope (Leica DMi8) equipped with an ACS APO 63X oil DIC immersion objective. Images were acquired using the LasX software (Leica), with a resolution of 1024 × 1024 pixels and a number of Z-stacks (step size of 0. 3 μm) encompassing the entire cell soma.
Data and statistical analysis Proteins identified in post-mortem mass spectrometry with less than two unique peptides were removed from the dataset (565 IDs). Similarly, proteins that were not identified in all samples were also excluded from the dataset (30 IDs). Following these data filtering steps, the dataset was analysis ready containing 5497 protein IDs. Reporter intensity value of each protein was log 2 transformed and the group median was corrected to account for minor loading variables. An average of technical replicates per protein were counted for each group, then sALS and C9 + ve group were normalized to the control group providing the fold change value for each protein  [1]. To visualize protein distribution by volcano plot, p value for each protein was generated using multiple t test with the original FDR method of Benjamini and Hochberg, then p values were -log 10 transformed.
Ratiometric values were generated by normalizing the average reporter intensity value of each protein of sALS and C9 + ve data to control than 1/group median corrected. Up -or down-regulated proteins were identified as a threshold of 20%-fold change (≥ 1. 2 or ≤ 0. 8 ratio change)  [38]. This fold change threshold was chosen as post-hoc validation of protein expression with western blotting can be conducted with these levels of protein change. Cell type-specific enrichment analysis of the post-mortem dataset was performed using different mouse and human cell type-specific gene clusters in the spinal cord  [8, 87]. Clusterspecific proteins were selected and visualized by heat map using Z-scores. To find synapse-specific TDP-43 interactors, a previously published TDP-43 whole-cell interactome dataset  [69]  was aligned with our post-mortem dataset.
After the initial overlay, proteins with synaptic functions were selected using the Gene Ontology (GO) "Synaptic" term, revealing 854 protein IDs which were displayed by heat map. Downstream statistical analysis of hiPSC-based proteomics and phosphoproteomics was performed in R (version 3. 6. 0). The R package proteus (version 0. 2. 14) was used to analyze MaxQuant's Proteomics output file "proteinGroups. txt" and the phosphoproteomics output file "Phospho (STY) Sites. txt". Differential expression (DE) analysis was performed with the R package Limma (version 3. 42. 2) outside of the package Proteus. As a cut-off for statistical significance, a nominal p value < 0. 05 or adjusted p values (adj. P. Val. ) < 0. 05 were chosen. All data were quantile normalized to account for the variation of intensity between samples followed by log2 transformation.
For graphical visualization, we used a volcano plot to show statistical significance -log10(p-value) versus log2 FC. Gene-centric analysis was performed by pooling the data obtained from mutant lines with mutations in the same gene and using as a significance cutoff 20%-fold change. For the identification of KEGG pathways and Gene Ontologies (GO) that may be altered we used the g:Profiler (https:// biit. cs. ut. ee/ gprofi ler/ gost) or ShinyGo v0. 75 online tools with the specification of Homo Sapiens datasets. The protein IDs corresponding to the DE phosphosites and proteins with nominal p-val < 0. 05 were copied into the g:Profiler tool. Homo sapiens was selected as the species and for the advanced options the following parameters were considered: only annotated genes, g:SCS threshold, 0.
05 threshold and ENTREZGENE_ACC; before clicking on Run query. Enriched terms were downloaded and plotted using the ggplot2 package for R version 4. 1. 2 or GraphPad Prism (version 9. 5. 0). Venn diagram calculation and plotting were generated using the free online software of the Van de Peer Lab (https:// bioin forma tics. psb. ugent. be/ webto ols/ Venn/). Heatmaps were plotted using hierarchical clustering with Euclidean distance by the heatmap. 2 modules of the gpltots package of R or the Partek Flow softwares. Z-score was calculated as described before  [38]. Protein-protein interaction networks were generated with the OmicsNet 2. 0 software  [91]. Kinome analysis was performed using the SELPHI tool  [56]  and the representation of the kinome tree was made using the KinMap tool (http:// www.
After sample quality control by total lysate staining to rule out major differences in tissue and protein degradation (Supplementary Fig. 1b , online resource), we followed a previously established pipeline  [38]  to analyze the total synaptic proteome using TMT-based mass spectrometry (Fig. 1a ). We performed internal quality controls to ensure that the data obtained with mass spectrometry were truly representative of the original samples. First, we calculated the ratiometric change in the expression of PSD95, RTN, TDP-43, SERPINA3 and SYP based on western blot experiments. These were strategically selected as they represented proteins that either decreased, increased or did not change in ALS samples compared to controls, allowing us to confirm a dynamic range of protein alterations. Afterwards, we plotted these values against the proteomics ratiometric data and found a significant correlation (R 2 = 0.
9515, p = 0. 0001; Supplementary Fig. 1c , online resource) between the two methods, suggesting the proteomics data accurately reflects changes within the original synaptic sample. Interestingly, the expression level of SERPINA3, TDP-43, RTN3, PSD95 and SYP were comparable in both ALS groups, suggesting similar disease-associated changes within this select panel of proteins, despite the different genetic background (Supplementary Fig. 1d , online resource). In our MS experiments, we identified 5495 unique proteins present in all samples, 87% of which (4752 proteins) overlapped with a previous proteome obtained from total spinal cord lysate (Supplementary Fig. 2a , online resource)  [54]. In addition, 85% of our dataset overlapped with 3 independent synaptic databases (SQLite, SynaptomeDB and SynGO; Supplementary Fig.
2b , online resource)  [26, 36, 57, 75]. Together, this extensive bioinformatic analysis confirmed the strong synaptic enrichment in our proteomic data. By performing an enrichment analysis of the whole dataset using the ShinyGO online platform, we identified a significant overrepresentation of pathways linked to metabolism and neurodegenerative disorders (Supplementary Fig. 2c , online resource). In addition, GO analysis revealed that the identified proteins were mainly associated with mitochondria, synapse, vesicles, protein transport and localisation, as well as molecule binding (Supplementary Fig. 2d , online resource), further confirming the strong enrichment 1 3 1 3 of synaptic material in our samples. Strikingly, we also identified the synaptic presence of 39 proteins coded by ALS pathogenic genes, whose levels were largely comparable in C9 + ve and sALS samples (Supplementary Fig.
We overlapped our dataset with publiclyavailable single-cell transcriptomics  [8, 87]  and identified cell-type specific markers revealing that the majority of the proteins specific for the different neuronal subtypes were indeed present in our post mortem specimens (Supplementary Fig. 3a-d , online resource). Importantly, we also could differentiate between alpha (vulnerable) and gamma (resistant) MNs  [26, 36, 57, 75]  (Supplementary Fig. 3e , online resource). This suggested that the selective loss of specific neuronal subtypes had a minor contribution to the synaptic alterations highlighted by our experiments and that the similarities between C9 + ve and sALS synaptome arise from molecular alterations rather than MNs death. The majority of the up-regulated proteins (1094) were indeed shared among both ALS groups (Supplementary Fig.
Given the fact that this protein is an important regulator of synaptic proteins and dynamics  [84] , we first searched for TDP-43 targets within our synaptic proteome. By aligning our dataset with a previously identified cluster of TDP-43 neuronal targets  [69] , we found that the majority of the proteins (4364) detected in our data could be potentially regulated by TDP-43. Within this large cluster, 864 molecules had a synaptic annotation and were significantly altered in the ALS groups (Supplementary Fig. 6a , online resource). In confirmation of the similarity between the C9 + ve and sALS synaptome, these 864 proteins displayed similar trends in both disease groups and the largest portion (including UNC13A) were down-regulated (Supplementary Fig. 6b  and Supplementary Table  2 , online resource).
After Tandem Mass Tag (TMT) labelling, proteome was examined using liquid chromatography, mass spectrometry-based quantification and identification (LC-MS/MS). Collected MS data were analysed using MaxQuant proteomics software. b Heatmap shows genotypic-specific protein changes using hierarchical clustering with Euclidean distance. c The number of up-and down-regulated proteins were counted using the control normalized ratiometric values on a threshold of 20% ≤ change in each direction. d, e Bioinformatics analysis revealed the top enriched KEGG pathways and Gene Ontology (GO) terms using the up-and down-regulated protein IDs. Colorized dot plots and bar graphs were created based on fold enrichment and gene counts/pathway. f Synaptic enrichment analysis using down-regulated protein IDs by the SynGO database shows enhancement in presynaptic terms including synaptic vesicle cycle ◂ 1 3 might be mainly affected, we analyzed the down-regulated proteins using the online synaptic protein database, SynGO.
Integration of post-mortem and hiPSC-derived MNs synaptosomal data identifies pathologic alterations in the vesicle release machinery Analysed independently, it was apparent that our postmortem and hiPSC synaptic data were revealing similar changes in presynaptic machinery. We, therefore, aimed to more accurately identify convergent synaptic alterations between the post-mortem and the hiPSC proteomes. First, we assessed the similarities across datasets without considering the different pathogenic mutations. The majority of the molecules identified in the in vitro model were also present in the post-mortem samples, even though we could detect a larger number of synaptic proteins in the latter ones (Supplementary Fig. 12a , online resource). HiPSC-derived MNs have a synaptic transcriptome closer to the embryonal one  [74]  and, accordingly, our analysis of the significantly altered proteins in the ALS groups highlighted a separation between in vitro and post-mortem samples (Supplementary Fig.
12 g , online resource). Despite the heterogeneity of this combined dataset, our data does reveal a consistent alteration of the presynapse, suggesting that dysfunctional synaptic vesicle machinery might play a crucial role in ALS. To gain more precise information on this aspect of the pathology and reduce some of the noise introduced by a diverse set of genetic backgrounds, we focused on the ALS C9orf72 genotype by comparing this specific sub-group of the ALS MNs to the post-mortem samples. Again, the majority of the synaptic proteins identified in vitro were also detected in the post-mortem samples (Fig. 3a ) and the hierarchical clustering separated the cultured cells from patients´ biopsies (Fig. 3b ). Enrichment analysis highlighted a significant representation of KEGG terms associated with neurodegeneration and others similar to those found when comparing the two larger, collective datasets (Fig.
3  Alignment of the human post-mortem and hiPSC c9orf72 cell line proteomics dataset reveals synaptic vesicular dysfunction. a C9orf72 hiSC cell line ALS C9orf72 shows 89% overlap with the human post-mortem dataset. b Heatmap demonstrates ALS group-specific protein changes using the 2501 congruent proteins. c-e Aligned dataset were used for further enrichment investigation. Based on fold enrichment and gene counts (dot size) top 10 KEGG pathways (c) were visualized by colorized dot plots. Different Gene Ontology (GO) terms (d-e) plotted by fold enrichment and the bar graph is colorized by gene number/pathway. f Synaptic enrichment analysis was created using SynGO database, revealing enriched synaptic functional terms such as synaptic vesicle cycle, presynaptic endocytosis and postsynaptic density.
4d ). We also found a significant enrichment in kinases involved, among others, in axon guidance, autophagy and mTOR pathway, as well as ErbB signaling (Fig. 4e ). When we compared the datasets associated with the ALS and Healthy genotypes, the phosphorylation of BAD at serine 118 was the top predicted target in mutant MNs (Fig. 4f ). In addition, analysis based on the phosphosites identified only in the disease group predicted phospho-c-Jun Ser63 as the most phosphorylated site, mainly as a target of mitogenactivated protein kinase (MAPK) family members (Fig. 4g ). We then investigated the levels of the kinases predicted with SELPHI in our post-mortem samples and found that 6 of them were up-and 8 down-regulated in both sALS and C9 + ve samples (Supplementary Fig. 18 , online resource). In particular, MAPK14 showed the highest up-regulation, suggesting a mechanistic link between synaptic alterations, c-Jun phosphorylation and neuronal stress.
5b ), suggesting a neuroprotective effect exerted by the treatment in all the mutant lines. In fact, treating ALS-related MNs with DHA significantly reduced the leakage of LDH (Supplementary 1 3 Fig. 25a , online resource), as well as the expression of the pro-apoptotic genes AEN, CASP3 and CFLAR (Supplementary Fig. 25b-d , online resource), whose expression was predicted to be controlled by Jun using the Harmonizome software  [62]. Accordingly, DHA-treated cultures displayed a significantly higher MN survival than those exposed to vehicle (Fig. 5c ). Thus, even though with different magnitudes (Fig. 5d ), DHA rescued the synaptic levels of SYP and the pro-apoptotic activation of the transcription factor c-Jun in a heterogeneous cohort of fALS-related MNs, providing crucial information on how convergent pathomechanisms might be beneficially targeted across the ALS spectrum.
These novel datasets mapping the spinal synaptic proteome in humans highlighted dramatic synaptic alterations in ALS. Interestingly, the magnitude of synaptic aberrations occurring within the spinal cord appears to be significantly larger than in cortical areas of ALS brains with comparable approaches  [38] , confirming the high vulnerability of this region. Because of this, it cannot be excluded that the selective death of MNs (and general neuronal sufferance) occurring within the spinal cord of ALS patients might at least partially contribute to the dramatic loss of synaptic contacts here described. Still, our data indicate that all cell types within the spinal cord are currently identifiable by proteomics and, most important, are present in our post-mortem samples. This strengthens our theory that the synaptic changes we identified in this study are mainly linked to molecular alterations within remaining synapses, rather than neuronal loss.
Furthermore, this approach prevents our ability to align synaptic alterations with the individual presence of neuropathology and its severity. For example, we discovered that several synaptic proteins whose levels are altered in ALS are also potentially under the regulatory control of TDP-43. It would be interesting to assess, at the individual level, whether the expression levels of these proteins were more severely altered in patients with more severe pTDP-43 neuropathology. Lastly, the hiPSC lines used in this study covered a large portion of the familial ALS spectrum, but these were not originated from the same patients we obtained post-mortem samples from and we also did not have access to sporadic ones to fully match the in vivo and in vitro datasets.
Acknowledgements The authors wish to thank the donors and their families for the generous tissue donations and the  Edinburgh Brain Bank  team for processing and providing material, namely its director  Professor Colin Smith  and manager  Chris-Anne McKenzie. The authors are thankful to  Maria Manz  and  Sabine Seltenheim  for their valuable technical support. The authors want also to acknowledge the great support received from  Silke Wahl  with sample preparation for MS experiments.
Data availability The datasets generated and/or analysed during the current study are available in the PRIDE repository and are available with the dataset identifiers PXD041543 (hiPSC-derivewd MNs) and PXD042617 (post mortem samples). Open Access This article is licensed under a Creative Commons Attribution 4. 0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.
Introduction Advances in natural language processing (NLP) have been driven mainly by scaling up the size of pre-trained language models, along with the amount of data and compute required for training  (Raffel et al. , 2020; Radford et al. , 2019; Rae et al. , 2021; Fedus et al. , 2021; Hoffmann et al. , 2022). However, these are not the only factors to determine a model's downstream performance, as the model's architecture and training objective are also important. He et al. (2021b)  showed that we can improve a model's performance by using disentangled attention, which uses two vectors to represent a token, one for position and one for content. He et al. (2021a)  later showed that performance could be further improved by using ELECTRA's  (Clark et al. , 2020)  self-supervised and sample-efficient replaced token detection objective.
Pre-trained French Language Models. Current language models available for French are either trained using Masked Language Modeling (MLM) or Causal Language Modeling (CLM). CamemBERT  (Martin et al. , 2020)  and FlauBERT  (Le et al. , 2020)  are two of the most popular contemporary French models, both trained with masked language modeling. Other models include FrALBERT  (Cattan et al. , 2021) , a French version of ALBERT  (Lan et al. , 2020) , LePetit  (Micheli et al. , 2020)  which is a small version of CamemBERT, and D'AlemBERT  (Gabay et al. , 2022) , a RoBERTa  (Liu et al. , 2020)  based language model targeted towards Early Modern French. BARThez  (Kamal Eddine et al. , 2021)  is a sequence-to-sequence model trained with BART's objective  (Lewis et al. , 2020), and PAGnol (Launay et al.
CAMEMBERTA: Methodology The following section details our proposed architecture and pre-training objective, along with descriptions for the downstream tasks. Architecture CAMEMBERTA is based on the DeBERTaV3  (He et al. , 2021b)  architecture which uses two vectors to encode the word and its position, with the premise being that the relative position of a word pair should also directly affect the computed attention weights. The V3 version optimizes the initial DeBERTa architecture by sharing the relative position embedding projection layers across all the encoder layers, and by adding a convolution layer aside the first encoder layer. 3  We use a base model configuration with 12 layers and 12 attention heads, 768 hidden dimensions with 32k for vocabulary size. Training Objective We follow the DeBER-TaV3  (He et al.
Downstream Evaluation We compare our models, CamemBERT CCN et , and CamemBERT 30% , on a diverse set of French downstream tasks and datasets, namely: Question Answering (QA) on FQuAD 1. 0  (d'Hoffschmidt et al. , 2020) , Part-Of-Speech (POS) tagging and Dependency Parsing on GSD  (McDonald et al. , 2013) , Rhapsodie  (Lacheret et al. , 2014) , Sequoia  (Candito and Seddah, 2012; Candito et al. , 2014)  in their UD v2. 2 versions and the French Social Media Bank 7   (Seddah et al. , 2012) , Named Entity Recognition (NER) on the 2008 version of FTB  (Abeillé et al. , 2000; Candito and Crabbé, 2009)  with NER annotation by  Sagot et al. (2012) , and the FLUE benchmark  (Le et al. , 2020). We use the dataset splits as provided by their respective authors, and we finetune using welltested scripts from the Hugging Face Transformers library and the HOPS parser  (Grobol and Crabbé, 2021).