dataesr/scientific-paragraphs-categorization huggingface dataset:
To add the trauma mechanism to the data collected by the OSCOUR network, a manual classification by health professionals would be time consuming and require multiple resources. Given the nature of the data (free text, unstructured, and containing abbreviations) to be processed and the objective (classification), artificial intelligence with deep learning, particularly automatic language processing, seems to be indicated. Natural language analysis has seen a recent breakthrough with the introduction of deep learning, in particular, the transformer architecture. Introduced in 2017 by Google and proposed in the article "Attention is All You Need" by Vaswani et al  [10] , transformers have an architecture that allows the implementation of a mechanism for processing the sequence of tokens (a token is an instance of a sequence of characters in a particular document that are grouped together as a semantic unit useful for language processing) that form a sentence in a self-attentive manner, that is, relating each of these tokens to each of the others in the sentence.
In other words, the text encoder converts text into a numeric representation. On many tasks, including text classification, its performance is systematically superior to that of the convolutional and autoregressive models used until then  [11]. French derivatives of the BERT model such as FlauBERT  [12]  and CamemBERT  [13]  have been trained on very large and diverse French corpora. FlauBERT is a French BERT trained on a very large and heterogeneous French corpus. Models of different sizes were trained using the Jean Zay supercomputer of the Centre National de la Recherche Scientifique; there are 3 sizes: small (54 million parameters), base cased (138 million parameters) and uncased (137 million parameters), as well as large (373 million parameters). CamemBERT is based on RoBERTa  [14] , which is an evolution of BERT in several aspects, including the use of the masked language model as the sole pretraining objective.
Recent studies have shown the effectiveness of transformers in classification tasks for EHR free-text data such as ICD coding  [24, 25] , phenotyping  [26] , and readmission prediction  [27]. Therefore, within the framework of the TARPON (Traitement Automatique des Résumés de Passage aux urgences dans le but de créer un Observatoire National) project, which aims to demonstrate the feasibility of setting up a national observatory of trauma, we propose here to compare the performances of several transformer models in the classification of ED visits for trauma based on clinical notes from the adult ED of the Bordeaux University Hospital. We compared the transformers FlauBERT, CamemBERT, BelGPT2, and a French GPT-2 model pretrained on a domain-specific corpus called GPTanam with term frequency-inverse document frequency (TF-IDF)/support vector machine (SVM), which was used as a baseline model.
Models and Experiment Settings The models selected for comparison and freely available as open-source content were a traditional machine learning model (baseline model) with TF-IDF/SVM couple as well as 3 transformer-type models pretrained on French corpora: CamemBERT  [13] , FlauBERT  [12] , and BelGPT2  [18]. We then chose the best performing model and applied a supplementary step of self-supervised training with the remaining 306,368 unlabeled clinical notes. This model is called here as GPTanam. Table  1  lists the size and configuration of each transformer model. For TF-IDF, tokenization was performed using the National Language Toolkit package (version 3. 6. 6; NLTK)  [29] , and linear support vector classifier was applied using scikit-learn (version 0. 24. 1) [30]. The most frequent words (eg, "that," "he," and "the") were removed.
Tokenization was performed using SentencePiece  [31]  for CamemBERT, Byte-Pair Encoding for FlauBERT, and a byte-level Byte-Pair Encoding for both GPT-2 models  [32]. The data were cleaned using regular expressions with the re package in Python (version 3. 7). Unicode normalization was performed in the 8-bit Universal Character Set Transformation Format. The linear support vector classifier parameters were as follows: tolerance=1 × 10 -5 , penalty=l2, loss=squared hinge, dual optimization=true, C=1. 0, multiclass strategy=one versus rest, verbose=0, and a maximum of 1000 iterations. For all 3 transformers, the optimizer was AdamW, with an epsilon of 1 × 10 -8 , and the maximum length was 512. GPTanam had training and evaluation batch sizes of 5 and a learning rate of 2 × 10 -5. For FlauBERT and CamemBERT, the batch size was 16 for training and 20 for evaluation, and the learning rate was 5 × 10 -5.
Self-supervised Learning and Fine-tuning Phase Considering the GPTanam model, the first step comprising self-supervised learning was performed with 306,368 clinical notes with 1 epoch  [33]. For all the models, a random sample of 80. 80%  (18,166/22,481)  of the clinical notes labeled as trauma was dedicated to supervised learning. This data set was divided into a training sample (14,532/18,166, 79. 99%) and a validation sample (3634/18,166, 20%) in an 80/20 ratio. We trained each model 9 times with different seeds on 7 epochs for CamemBERT and FlauBERT and 5 epochs for BelGPT2 and GPTanam. To obtain a single prediction for the 9 different executions of the chosen epoch (based on the maximum validation micro F 1 -score) for each model, a vote was taken.
Fine-tuning the Performance of Models Unlike statistical methods such as TF-IDF, the supervised fine-tuning of transformer models is time consuming and can be greatly accelerated by the use of graphics processing units. The self-supervised fine-tuning step for the GPTanam model required approximately 12 hours. At that point, GPTanam could generate artificial clinical notes, as seen in Multimedia Appendix 3, that could not be easily differentiated from the original ones. One epoch of supervised fine-tuning required 15, 16, 19, and 18 minutes for CamemBERT, FlauBERT, BelGPT2, and GPTanam, respectively. When looking deeper into each transformer model's F 1 -scores on the validation data set, Figure  1  shows that CamemBERT reached its maximum F 1 -score (0. 873) at epoch 6, FlauBERT achieved an F 1 -score of 0.
Similarly, the treatment of medical concepts and abbreviations remains an area for improvement, as not all EDs use the same abbreviations in the same context. The use of ontologies developed in the field of emergencies could constitute an area for improvement. Transformers have also recently been tested for the identification and replacement of abbreviations, with good results for BERT  [42, 43] ; however, there has not yet been a test on data from a mixture of common language and medical terms in French. In addition, because the authors who proposed the CamemBERT model did not compare the performance of different models from the OSCAR, CCNET, and Wikipedia data sets in a classification task, a future study could compare the different sets in our database in this regard. While we have only used the basic models of CamemBERT, FlauBERT, and GPT-2, it would be appropriate to test the different sizes of pretraining data sets on a classification task as well as the different sizes of models.
Indeed, Martin's  [44]  team has shown that the standard CamemBERT model (110 million parameters) trained on all 138 GB of OSCAR text does not massively outperform the model trained "only" on the 4 GB sample in morphosyntactic labeling, syntactic parsing, named entity recognition, and natural language inference. One perspective considered is to test different models of French transformers that have been released since CamemBERT, FlauBERT, and BelGPT2 such as Pagnol and BARThez.
Acknowledgments This work was carried out within the framework of the  TARPON (Traitement Automatique des Résumés de Passages aux urgences pour un Observatoire National)  project led by the  Inserm team Injury Epidemiology  (project leader  E Lagarde ) and the emergency department of the  Bordeaux University Hospital  in collaboration with the Statistics In System biology and Translational Medicine team, managed by  Inria and Inserm. This project is the winner of the second call for projects of the Health Data Hub,  Grand Défi  " Improving medical diagnosis through Artificial Intelligence " and  Bpifrance. This study was conducted within the framework of PIA3 (Investment for the Future; project number  17-EURE-0019 ). The authors would like to thank all the members of the labeling team. The authors would also like to thank the  University Hospital of Bordeaux  for providing logistical support, which allowed the authors to access and analyze the data needed for the manuscript in such a short period.
Whole Genome Sequencing (WGS) The whole genome of the ΔcorAΔmgtA colony variants (SCV and LCV, S1 Table ) and the parental strain ATCC14028s was sequenced at the Novogene Corporation Inc, using an Illumina NovaSeq sequencer. Total genomic DNA was extracted using the FastDNA spin kit for soil (MP Biomedicals). DNA fragmentation, WGS libraries preparation and sequencing was performed by Novogene using a 2 × 150 nucleotides paired-end strategy and the quality of the resulting data was controlled. Library adaptators of paired-ends read were trimmed with Cutadapt version 2. 10 (https://doi. org/10. 14806/ej. 17. 1. 200) and then mapped to the reference genome of Salmonella enterica subsp. enterica serovar Typhimurium (GenBank accession numbers CP001362 and CP001363 for the virulence plasmid and chromosome, respectively) using the Burrows-Wheeler Alignment tool, BWA version 0.
7. 17 with the BWA-mem algorithm. SAMtools version 1. 13 and Picard tools version 2. 23. 3 (http://broadinstitute. github. io/ picard) were then used to filter, sort, convert SAM files and assign read groups and mark duplicate reads. Single-nucleotide polymorphism (SNPs) and insertion/deletion (indels) positions were called using Genome Analysis Toolkit version3. 1-130 according to the GATK Best Practices for haploid genome (VariantFiltration, QD <2. 0, LowQD, ReadPosRankSum<-8. 0, LowRankSum, FS >60. 0, HightFS, MQRankSum<-12. 5, MQRankSum, MQ <40. 0, LowMQ, HaplotypeScore >13. 0, HaploScore parameters were used to filter SNPs and indels). Twentyone SNPs and eight INDELs were identified in the genome of the parental strain (VF6910, S1 Table ), compared to the published sequence of ATCC14028s  [28].
For each sample, peptides were loaded on an home-made 38 cm capillary column picotip silica emitter tip (75 μm inner diameter) with C18 resin (1. 9 μm particles, 100 Å pore size, Reprosil-Pur Basic C18-HD resin, Dr. Maisch) after an equilibration step in 100% solvent A (H2O, FA 0. 1%). Peptides were eluted with a multi-step gradient using 2 to 7% solvent B (ACN 80%, FA 0. 1%) during 5 min, 7 to 23% during 120 min, 23 to 45% during 20 min and 45 to 95% during 7 min at a flow rate of 250 nL/min over 167 min. Column temperature was set to 60˚C. MS data were acquired using Xcalibur software using a data-dependent Top 10 method with a survey scans (300-1700 m/z) at a resolution of 70,000 and a MS/MS scans (fixed first mass 100 m/ z) at a resolution of 17,500. The AGC target and maximum injection time for the survey scans and the MS/MS scans were set to 3E6, 50 ms and 1E6, 60 ms respectively.
The isolation window was set to 1. 6 m/z and normalized collision energy fixed to 28 for HCD fragmentation. We used a minimum AGC target of 1E4 for an intensity threshold of 1. 7E5. Unassigned precursor ion charge states as well as 1, 7, 8 and >8 charged states were rejected and peptide match was disable. Exclude isotopes was enabled and selected ions were dynamically excluded for 45 seconds. Data analysis. MaxQuant. Raw data were analyzed using MaxQuant software version 1. 6. 6. 0  [32]  using the Andromeda search engine  [33]. The MS/MS spectra were searched against the Salmonella typhimurium strain 14028s UniProt database (5,369 entries the 17/01/ 2021) and against a small protein database containing 5 proteins. Andromeda searches were performed choosing trypsin as specific enzyme with a maximum number of two missed cleavages.
[34]. Unique and razor peptides included modified peptides, with at least 2 ratio count were accepted for quantification. PEAKS studio. Data were analysed using PEAKS Studio 7. 0  [35]. De novo sequencing and database search of peptides was performed with common parameter settings such as trypsin digestion, parent mass tolerance 5 ppm and fragment mass tolerance 0. 02 Da. Possible modifications included carbamidomethylation (Cys, fixed), oxidation (Met, variable) and Nter acetylation (variable). For de novo sequencing ALC score was set �50% and for database search the same database as MaxQuant was used. A cutoff of 1% false discovery rate (FDR) was applied at PSM level. Statistical analysis. To find the proteins more abundant in one condition than in another, the LFQ intensity values from Maxquant were compared.
mle function of the R package imp4p  [37]. Statistical testing was conducted using a limma t-test thanks to the R package limma  [38]. An adaptive Benjamini-Hochberg procedure was applied on the resulting p-values thanks to the function adjust. p of the cp4p R package  [39]  using the robust method described in  [40]  to estimate the proportion of true null hypotheses among the set of statistical tests. The proteins associated to an adjusted p-value inferior to an FDR (false discovery rate) level of 1% and an absolute log2 (fold-change) superior to 1 have been considered as significantly differentially abundant proteins. Finally, the proteins of interest are therefore those which emerge from this statistical analysis supplemented by those which are present from one condition and absent in another ("ID Table" in the S1 Dataset).
Functional analysis GO terms associated to each identified protein were retrieved from Uniprot database (www. uniprot. org) and hypergeometric tests were performed to test the overrepresentation hypothesis for each GO term using R software. All the proteins identified in our mass spectrometry experiments have been used as background for the hypergeometric tests. A significantly low p value means the proportion of proteins related to a GO-term is significantly superior in the considered list than in this background (S15 Fig). STRING database (http://string. embl. de/) was used to visualize protein-protein interactions. The active interaction sources were Text mining, Experiments, Database, Co-expression, Neighborhood, Gene fusion, and Co-occurrence, with a required minimum combined score of major confidence (0.
The effect of the ΔcorA mutation on the proteome and motility of Salmonella is majored in the absence of PhoP To determine why combination of the ΔcorA and ΔphoP mutations, but not the mutations alone, impaired growth, we asked whether combination of these mutations was accompanied by a differential protein cargo content. A global quantitative proteomic analysis was carried out to compare the proteome of the wild-type strain, and the ΔcorA, ΔphoP and ΔcorAΔphoP mutants, using five biological replicates of each strain (S1 Dataset and S17 Fig)   S7 ). A total of 2688 proteins were identified ("ID Table" in S1 Dataset), representing at least 50% of the reference proteome of strain ATCC14028 (UniProt-Taxon ID 588858, https://www. uniprot. org/taxonomy/588858). The remaining proteins might not be expressed under the growth condition employed in this study, or they might be expressed at levels too low to be detected.
Also, membrane proteins are less easily identified by MS-based proteomics than soluble proteins. Combination of the ΔcorA and ΔphoP mutations had a profound effect on the Salmonella proteome. Almost five hundred proteins of various functions were found differentially abundant between the ΔcorAΔphoP mutant and the wild-type strain (S1 Dataset). Then, enrichment analyses were performed to assess which relevant functional or mechanistic biological processes were significantly enriched in the lists of proteins differentially abundant between the mutants and the wild-type strain. Interestingly, it appears that many proteins less abundant in the ΔcorAΔphoP mutant than in the wild type strain and not found less abundant in the ΔcorA and ΔphoP strain are related to flagella formation, chemotaxis, T3SS-dependent secretion, and cobalamin biosynthesis  (Fig 6).
A large number of proteins involved in these pathways were affected by the ΔcorA and ΔphoP mutations combined, but not alone (S1 Dataset and S17 Fig). PhoP has been reported to repress flagellum-encoding genes, thus preventing flagellummediated motility  [10]. In our experimental conditions, there was no positive effect of the ΔphoP mutation alone on motility and flagellum synthesis ( Under magnesium starvation, PhoP represses genes of SPI-1, encoding the T3SS-1 implicated in invasion of epithelial cells, which may explain why a PhoP-null mutant enters epithelial cells better than the wild type strain  [10]. In contrast, PhoP promotes transcription of genes encoded by SPI-2, encoding the T3SS-2 implicated in macrophage survival and virulence  [10]. Genes encoding components of SPI-1 and SPI-2 were repressed in a microarray study with a Salmonella corA mutant grown to log phase in rich medium  [17].
In our experimental conditions, functional categories of proteins involved in secretion via T3SS were not found significantly enriched in the ΔphoP and ΔcorA mutants, but they were enriched in the ΔcorAΔphoP strain (Figs 6 and S17 and S1 Dataset). Indeed, several proteins secreted or involved in secretion via T3SS were less abundant in the ΔcorAΔphoP mutant, an outcome which might be explained by the fact that HilA and SsrB proteins, the major transcriptional activator of SPI-1 and SPI-2 genes, respectively  [54, 55] , were less abundant in the ΔcorAΔphoP mutant (S1 Dataset). Impact of the ΔphoP and ΔcorA mutations alone. A large set of proteins was affected by the ΔphoP mutation alone, confirming that PhoP was functional in the wild-type strain grown to stationary phase in LB supplemented with magnesium 10 mM (S1 Dataset).
Some of these proteins are known to be positively controlled by PhoP under magnesium starvation (PagC, RstA, IraP, ArnB, SseB, SlrP, BasR, S1 Dataset)  [10]. The response to Mg 2+ -starvation orchestrated by PhoPQ is complex, with transcription of individual genes being induced at specific stages of the stress  [56]. The pattern of transcription activation by PhoP is not dictated solely by the affinity of phosphorylated PhoP for its target promoters but it is also influenced by additional regulatory mechanisms and modulation of PhoP and PhoQ activity  [10, 57]. Under the magnesium-proficient growth conditions used in this study, PhoP is probably activated through another environmental signal than magnesium starvation. PhoP is known to respond to other signals, including Ca 2+ , low pH and antimicrobial peptides  [10]  and additional signals are probably to be discovered, especially when the extracellular concentration of magnesium is high.
A major role for PhoP is the control of the cell envelope composition and regulation of LPS modifications which help bacteria to evade the immune system or confer resistance to antibacterial compounds or cation toxicity by lowering the negative charge of the membrane  [10]. With regard to magnesium, modifications affecting lipid A phosphate residues release Mg 2+ neutralizing ions, allowing the membrane to be a Mg 2+ reservoir that can be used under magnesium deprivation  [10]. Sixty-five proteins affecting membrane functions and LPS were more abundant in the ΔphoP mutant than the wild-type strain, indicating a cell wall remodeling in the absence of PhoP even when magnesium is available (S1 Dataset). The effect of combination of the ΔcorA and ΔphoP mutations on the composition of the cell envelope was even more drastic with more than one hundred proteins more abundant (S1 Dataset).
Several of these proteins are involved in transport activities and most are intrinsic components of membrane (S1 Dataset). The effect of the ΔcorA mutation on Salmonella proteome was more subtle than that of the ΔphoP mutation, with fifty-three proteins found differentially abundant with the wild-type and involved in various functions (S1 Dataset). Of note, among the fifty-three proteins affected by the ΔcorA mutation, proteins involved in pyrimidine and propanediol metabolism were enriched and significantly less abundant in the ΔphoP and ΔcorAΔphoP mutants (S1 Dataset and S17 Fig). Although the biological significance of these effects, common to the ΔphoP and ΔcorA mutations, must be determined, they suggest the existence of a certain level of functional redundancy between the CorA and PhoP systems under the experimental conditions used.
Acknowledgements This work was partially supported by  Normandie Université, Région Normandie , the  Centre National de la Recherche Scientifique ,  Université de Rouen Normandie ,  INSA Rouen Normandie ,  Labex SynOrg  ( ANR-11-LABX-0029 ), the graduate school for research  XL-Chem  ( ANR-18-EURE-0020 XLCHEM ) and  Innovation Chimie Carnot (I2C). This work has been funded by the  Agence Nationale pour la Recherche  ( ANR-21-PRRD-0001-01 ) and NovAliX. M. A. thanks the  Agence Nationale pour la Recherche  ( ANR-21-PRRD-0001-01 ) and  NovAliX  for a postdoctoral fellowship.
Zopiclone, like zolpidem, is under surveillance in France due to its potential for dependence, abuse and misuse. However, part of the narcotics regulation was implemented in 2017 in France for zolpidem only, which has led to an increase in the number of zopiclone consumers. The objective of this article is to present French addictovigilance data regarding the evolution of the abuse, dependence and misuse profiles of zopiclone. We used the following 3 data sources over two periods: the 3-year period before the regulatory measure regarding zolpidem (2014-2016) and the 3-year period after the regulatory measure (2018-2020): (i) Medic'AM, a public database which provides a good overview of the number of boxes of zopiclone reimbursed in France; (ii) all French cases of drug dependence or abuse reported by health professionals to the French Addictovigilance Network (FAN); and (iii) a national epidemiological tool based on the surveillance of falsified prescriptions.
, 2019). Furthermore, a latent class analysis based on French regional health reimbursement databases did not specify a class of problematic consumers for zopiclone, whereas it indicated a subclass of zolpidem users prone to abuse  (Victorri-Vigneau et al. , 2013). In addition to this epidemiological study, Rousselet et al. clinically identified a higher prevalence of physical and compulsive signs of dependence and harmful consequences of dependence for zolpidem compared with zopiclone  (Rousselet et al. , 2017). For the OSIAP tool, in 2014, while zolpidem was present on 38% of falsified prescriptions, thus becoming the most cited molecule, zopiclone was mentioned in only 10% of falsified prescriptions but was also reimbursed less frequently than zolpidem  (Jouanjus et al. , 2018; Rousselot et al.
Study Oversight In this national study, we used 3 data sources: (i) Medic'AM is a public database, provided by the French national health care system, that indicates the number of tablets reimbursed in France each month and for each reimbursable drug. Zopiclone is a reimbursable and strictly prescribed drug in France, so Medic'AM provides a good proxy for the exposure of the French population to this molecule, even though it is an indirect indicator. (ii) NOTS: we focused on the addictovigilance notification mentioning zopiclone as "suspected" reported by health professionals, collected by the FAN, analysed by medical pharmacologists working in the addictovigilance centres and recorded in the French national database. Those records include sociodemographic data, medical history and co-consumption data and data on the products consumed: route of administration, quantity, duration, search effects and method of obtention.