dataesr/scientific-paragraphs-categorization huggingface dataset:
Precise purities were obtained except for one case where the NMR signal was heavily overlapped with the one of another molecule. To the best of our knowledge, this work stands as the first example of using benchtop NMR in an integrated workflow for identifying, elucidating and quantifying NPS. This workflow can serve as an alternative to the use of GC-MS: It could allow forensic laboratories to efficiently characterize and quantify a wider panel of drugs and especially NPS. The present workflow and database work well for simple elucidation cases, for more difficult ones, the creation of a bigger database including more reference structures is needed. In addition, the proposed workflow could be combined with the current workflows already used by the police, by including GC-MS in the database to maximize its potential.
parameter is correctly adjusted, the spectrum of the unknown NPS should either yield a unique answer or deliver no match (i. e. , HQI 100 or 0). In some cases, however, the HSQC spectra recorded on a benchtop spectrometer could be limited by sensitivity due to the low natural abundance of 13  C nuclei. This is why 1D1  H NMR spectra were also incorporated in the workflow, offering a more sensitive, yet F I G U R E 2 Workflow for structural identification and elucidation. For the heteronuclear single quantum coherence (HSQC) comparison in the elucidation section, the looseness factor (LF) is incremented by x (a nonzero natural number) [Colour figure can be viewed at wileyonlinelibrary. com] less resolved technique to compare the spectral patterns of the unknown molecule with those included in the database.
is dedicated to cases where the HSQC comparison returns no match. In such scenario, one would try to pinpoint the closest structure present inside the database, in order to provide structural insights into the molecular structure of the unknown NPS. To this end, the HSQC comparison is performed again with less restrictive LF values which are incremented until one match is obtained. This can help in determining the family of the unknown structure or part of its skeleton. Finally, IR and 1D 1 H NMR can be used to refine the proposed structure which can be validated by comparison between the experimental HSQC spectrum and a 2D HSQC spectrum predicted by the software. Of course, full elucidation with a database can remain difficult, especially when considering the limited performance of benchtop NMR.
used in practice. All relevant data associated to these cases are given in supplementary information. Sample no. 1 gives one signal on the 1D 19 F spectrum. AMB-FUBINACA (HQI = 100), AB-FUBINACA (100), and MDMB-FUBINACA (100) were returned as matches by comparison with the fluorine spectra present in the database. It indicates that the structure is part of the cannabinoid family and more specifically of the FUBI-NACA subfamily. Then, HSQC comparison provided only one hit, namely, AB-FUBINACA (100). The structure was then easily validated.
ACKNOWLEDGEMENTS The authors acknowledge Magritek, and especially  Hélène Freichels , for her help with the autosampler and  Craig Eccles  for his help with the SpinsolveExpert software. The authors are also indebted to  Benoît Charrier  for his help in the maintenance of the spectrometers. Shahriar Jahanbakht  ( ACD/Labs ) is warmly acknowledged for his clear explanations about the various features of the  ACD/Labs  software. We thank the  Finnish National Bureau of Investigation  for giving us some samples to complete our database. The authors from the CEISAM lab acknowledge the  French National Infrastructure for Metabolomics and Fluxomics MetaboHUB - ANR-11-INBS-0010  (www. metabohub. fr) and the  Corsaire metabolomics core facility (Biogenouest). This work includes NMR experiments carried out the on the CEISAM NMR platform.
Acknowledgments: The authors are grateful to the  Société Mycologique de Strasbourg , and particularly to  Dr J. -F. Stambach  and  R. Wiest , for their help with preliminary mushroom identification. We also want to thank  Dr D. Blaha  from the  University Claude Bernard in Lyon , for providing the strain of S. epidermidis. D. Keller  from  UMR 7290 ,  University of Strasbourg , is gratefully acknowledged for cultivation of MDR strains.
Assemblies were finally deposited in the GenBank under the Bio Project number PRJNA751742 and the strain was labeled as PLV. Based on TYGS analysis we found that (when considering 16S rDNA sequence) the most related species/ strains were: Paenibacillus lautus (NBRC15380), Paenibacillus glucanolyticus (DSM5162), Paenibacillus lactis (DSM15596) and Paenibacillus ihbetae (JCM 31131T). Similar results were obtained when whole-genome tree was analyzed, however, Paenibacillus ihbetae was replaced by Paenibacillus solani (FJAT-22460) in the cluster of the most similar species (Fig. 1 ). To our knowledge, this is the first study to report the isolation of P. lautus from in vitro L. aestivum plants. Previously, Spina et al. 14  isolated bacteria belonging to the genus Bacillus from in vivo and in vitro L.
The obtained genomic DNA of the endophytic bacteria was used for WGS (whole genome sequencing). The sequencing library was prepared commercially using Illumina's Nextera XT DNA library kit, and quality was assessed by Qubit DNA Assay (Thermo Fisher; Waltham, Massachusetts, USA), Agilent Bioanalyzer 2100 (Agilent; Santa Clara, CA, USA), and qPCR at Applied Biological Materials Inc. (Richmond, BC, Canada). The obtained libraries were sequenced in a paired end 150 bp run on Illumina (San Diego, CA, USA) HiSeq 4000 platform. Raw sequencing reads were quality controlled using FastQC software (Babraham Bioinformatics), and filtered using Trim Galore software (Babraham Bioinformatics) along with adapter sequences removal. Cleaned reads were assembled into cotigs with Shovil v1. 1. 0 which is an ultra-fast implantation of Spades v3.
14. 0 algorithm  47. The obtained contigs were finally polished using Pilon 1. 23 software  48  and scaffolded using MeDuSa web server  49  against Paenibacillus lautus strain E7593-69 (GCA_003590055. 1; ASM359005v1) genome. Scaffold belonging to plasmid was identified by Nucleotide BLAST against the reference genome sequence. Contigs statistics were retrieved using Quast v5. 0. 2 software  50. The sequences were annotated during genome submission by the NCBI Prokaryotic Genome Annotation Pipeline (PGAP)  51  using Best-placed reference protein set (GeneMarkS-2+) v5. 1. Taxonomic analysis of the isolated strain and search for the most similar spices were done using TYGS web server  52. Both 16S rDNA (SSU) and whole-genome based phylogenic trees were constructed.
8. The experiment was established in three replications. About five grams of plants were placed in each bioreactor vessel, 60 plants for each combination were used. The plants were grown for various periods of time: 7, 14 and 28 days. After these culture periods dry weight (DW) was measured for the plants after lyophylization (FreeZone 6 Liter freeze dryer, Labconco, USA). The samples were also taken from each treatment for phytochemical and biochemical analyses. Alkaloids analyses in bacteria and plant samples. The alkaloids were extracted from dried plants and bacterial pellets as previously described by Spina et al. 14  , purified and analysed using GC-MS system QP2010-Shimadzu equipment (Shimadzu, Kyoto, Japan) operating according to Saliba et al. 's method  9. The identification of the alkaloids was performed by comparing the measured data with those of authentic compounds (galanthamine, lycorine) or with literature data as specified in the text.
Acknowledgements The authors thankfully acknowledge  François Dupire  from  MassLor Plateform of Laboratoire Lorrain de Chimie Moléculaire  ( L2CM UMR 7053 ) in  Lorraine University and MSc Sonia Kozaczka from University of Agriculture in Krakow, Poland. The research was financed from the subsidy of the  Ministry of Science and Higher Education of the Republic of Poland  awarded to the  University of Agriculture in Krakow , Poland in 2020 year. This research was also supported by the  MEAE  and  MESRI PHC   Polonium  project (DWM. WKE. 183. 10. 2018/AM,2018-2019) financed by the  Ministry of Science and Higher Education of the Republic of Poland , by the " Impact Biomolecules" project of the "Lorraine Université d'Excellence" (Investissements d'avenir- ANR -15-IDEX-04-LUE ) and by  Institut Jean Barriol, Université de Lorraine, France.
Data availability The datasets generated during the current study are available from the corresponding author on reasonable request. The genome-wide shotgun design of the endophytic bacterium has been deposited with DDBJ/ENA/ GenBank as part of accession JAIFIS00000000. The version described in this paper is version JAIFIS010000000. The raw sequence data are available [https:// www. ncbi. nlm. nih. gov/ search/ all/? term= JAIFI S0100 00000].
These outputs are then combined to build an efficient and performing system aiming at qualifying and extracting the information corresponding to the following questions: • What are the monitored drugs and indication mentioned in the document? • What are the sentences that mention an established relationship between a drug and an AE? • What are the entities recognized as Drug or Adverse Event? The identification of relevant documents regarding seriousness drug adverse reaction signals is then performed on the basis of this information and meta data available in the data source (Ex: date, journal, type of publication, etc. ). 4. A benchmark dataset for seriousness classification task based on PubMed literature sentences. After a review of related work, we describe the LiSA pipeline architecture and provide a high-level performance analysis of the proposed solution.
Related Work In most of the papers mentioned in this section, the focus is on Adverse Events (AE) detection and not on Adverse Drug Reaction (ADR), meaning that there is no specific detection of a drug associated with an adverse event. For the sake of clarity we will use, only in this part, the terms Adverse Drug Events (ADR) to indifferently designate AE or ADR. Adverse Drug Reaction detection plays a key role in drug-safety surveillance and has motivated the creation of various monitoring systems or databases. The FAERS  [11]  reporting system and Medwatch  [12] , a medical product for safety reporting, are the current official solutions provided by the FDA. Meanwhile, these tools are only based on declarative reports and not on systematic analysis of the biomedical literature or any web-based source to identify potential ADRs.
Several solutions have been proposed to perform biomedical literature monitoring in order to identify, filter and rank papers related to a specific domain or medical concept. For example, ASE  [13]  demonstrates the value of reference management, statistics, natural language summarizing to interactively select key papers. STELLAR  [3]  leverages data mining techniques to help researcher to identify, rank and recommend reference papers for a specific literature review. More recently, Schoot et al. [1]  proposed ASReview, an efficient active learning based-tool to perform systematic literature review and meta-analysis. As per today, only a small number of literature review systems relate to adverse drug reactions detection. Among them, the PV-OWL tool  [2]  was built to link different databases to obtain novel safety indicators (FAERS, PubMed, social media.
). The semi-automated pipeline published by Yeleswarapu et al. [14]  supports extracting ADR pairs from adverse events databases using statistical BPCNN algorithm for Natural Language Processing. Among other classical approaches commonly used in NLP, distributional semantics based on patterns of ADR co-reporting  [15] , Hidden Markov Models  [16]  or disproportionality analysis (DPA)  [17]  were already attempted to perform ADR detection. In 2012, Gurulingappa et al. [18]  published an open-source reference dataset and developed a dictionary-based algorithm for extraction of adverse drug events in PubMed literature  [19, 18]. Following the significant advances in natural language processing with deep learning, more recent publications have exploited these technologies to improve safety signal detection.
Query definition and document collection Most of the medical publications that mention adverse drug reaction are published and available through PubMed, a free archive of biomedical and life sciences journal literature and considered as a reference for biomedical publications. Some publications require a licensed access, but still provide a free version of the corresponding abstract. Therefore, PubMed was used as the main data source for literature monitoring to build the system. By construction, documents collection should be associated to a "query" which is composed of a combination search terms. A query contains a main drug, an optional second drug and optional indication. Consequently, LiSA collects all articles available on PubMed published on the last 6 years associated to a query through the PubMed API.
Only the main drug serves as keyword search to trigger the API. Other query terms (optional drug and indication) are only searched to tag the document if they are mentioned in it. Drug and indication could be expressed under various synonyms in biomedical literature. To ensure the comprehensiveness of data collection, every drug and indication term is conjointly searched with all its synonyms based on open-source molecule and disease classification. For drug, we used the Chembl database  [20] , and for indications the MedDRA  [21]  hierarchy. Table  1  gives an example of a query definition.
Data preparation To structure the documents, we propose a standard architecture able to accommodate any type of collected documents or data sources and adapted to natural language processing algorithms. As a matter of fact, raw documents cannot be processed directly by transformers and achieve a satisfying performance  [22]. They should be split into meaning units of limited number of tokens like sentence or short paragraphs. This process, called sentence tokenization, is performed with a pre-built algorithm (on common English language) from the package nltk and adapted with specific cases found in biomedical literature. Structured data is then formatted into 3 different tables: During the split, if a sentence is too short (between 4 to 10 words), it is concatenated with either the previous or the next one (only in case it is less than 20 words long) to reduce the risk of missing an AE-Drug relationship.
Deep learning The three main AI modules presented in this section are the core of LiSA. They correspond to 3 different NLP tasks which are computed in parallel for all sentences containing a monitored drug (as described in part 3. 2. 2). Once calculated and stored in the database the different information are used to filter and qualify the hit sentences and relevant documents as depicted in Figure  1. Details about the different pre-trained algorithms and their respective performance are not provided in this paper but will be described in a forthcoming publication. Figure  2 : Simplified machine learning architecture of the LiSA pipeline. Unlike other post processing tasks, the serious regex search task described in Section 3. 3. 4 is performed before deep learning inference and is not represented on the schema.
Drug-AE relationship classification To assess the association between a drug and an AE, we chose to rely on state of the art deep learning algorithms with attention-based mechanism (BERT). This family of algorithms is trained on very large corpora to build contextual embeddings and has been shown to perform extremely well in highly context-dependent prediction tasks, such as AE detection. The presence of a drug-AE causality relationship within a sentence was predicted with a two class ("has causality", "has no causality") sentence classifier, as defined in the ADE-Corpus-V2 dataset  [19]  used for training. This dataset contains more than 20 000 sentences extracted from PubMed and pre-labelled for drug-AE causality classification. In particular, the two classes are defined without prior knowledge of the entities corresponding to drug and ADRs.
The ADE-Corpus-V2 dataset was split into training, validation, and testing sets with the ratio of 8:1:1 and used to fine-tune several pre-trained algorithms and to select the most accurate one. In order to further increase prediction performance, we performed manual data augmentation based on badly predicted observations of ADE-corpus-V2. Typical treated case are sentences including a negation form, containing an unspecified adverse effect ("AEs", "TRAEs", "Serious adverse effects") or related to specific lexical fields. The score threshold allowing the algorithm to classify a meaning unit was also calibrated to maximize the balance between recall and precision (we used F1score and precision-recall curve for that purpose). In the production version of LiSA, every meaning units predicted class and score are stored in the meaning units table.
Named Entity Recognition (NER) LiSA is also supposed to identify the different entities found in a relevant document corresponding to a drug or an Adverse Drug Reaction. For this task, we used Named Entity Recognition (NER) pre-trained algorithms within the same family of algorithms built on BERT architecture. Using the same open source corpus, we fine-tuned and bench-marked several models for the task of identifying two different entities: drug and ADR. In the final pipeline, the entity detection is only applied on meaning units that successfully passed the drug-AE causality prediction with a score higher than the defined threshold. This pre-filtering step was made to reduce the inference computation time. As for the previous step, detected entities and associated scores are stored in the meaning units database.
Unlike common ADR detection, we did not found an open access dataset to train the seriousness detection algorithm. This problem was overcome by labelling 7776 sentences extracted from PubMed in three categories: "serious", "important medical event", "none" (a "serious" sentence being an "important medical event" sentence with a serious outcome). The labelling process was performed by medical reviewers. The third class "important medical event" was only added to have a more detailed labelled dataset for possible additional application in ADR detection. 917 sentences were used as a testing test and allowed to reach a performance at the state of the art after adjusting the score threshold. More concretely, this models yields a class and a score and is only calculated on meaning units that contains at least one drug entity and one ADR entity from the NER module.
Results Performance assessment was performed with two strategies: • Evaluate the results based on a train/test approach on different datasets for different tasks. Detailed benchmarks and result analysis are not described in this article, only the performance of the selected models is displayed in Table  2. • Evaluate the performance of LiSA from the perspective of medical reviewers (end users). For individual NLP tasks evaluation, we used a specific test dataset for each task. This test set was created by selecting 10% of available labeled data that remained unseen by the algorithm. For AEdrug relationship classification as well as NER, we used the ADE-corpus-v2 dataset. For seriousness classification, the test set was carved out of the manually labelled dataset mentioned in subsection 3. 3.
Sentence level evaluation of LiSA To assess the performance at sentence level, we chose to use the classic performance metric for binary classification: precision, recall and f1-score. To calculate those metrics, we retrieved all documents associated with a list of drugs and synonyms, as described in Section 3. 1 (the list of drugs is available in appendix). All documents were then fed into the LiSA pipeline to detect all positives sentences (hits) and their parent articles. The volume of documents and meaning units after every successive filter is available in Figure  5. In the absence of a benchmark dataset to evaluate the performance of serious ADR detection, we created the SADR dataset with the help of medical reviewers with the following procedure. We first collected all documents freely available on Pubmed that contains a drug in the list available in appendix , and only kept the sentences that explicitly contains one of the drugs (since its absence would inevitably make the sentence irrelevant).
These sentences were passed to the pipeline to get a prediction regarding the presence of a serious ADR. Then we asked medical experts to review the sentences and check whether the prediction was correct or not. In total, 1231 sentences from 988 unique documents were analyzed. In that sample, LiSA reached a performance of 88. 6 % in recall, 81. 1 % in precision and 84,7% in F1-score. The achieved performance makes LiSA a state of the art system in terms of safety signal detection for the use-case considered in as much as it is closed to the performance obtained in 10. Meanwhile, the task evaluated in this paper differs from the case of LiSA. Especially, there is no mention of a drug-AE relationship classification task. In addition there is no code available neither benchmark dataset from 10 that could have been used for direct comparison.
For benchmark purpose, we provide the test dataset used to assess LiSA's performance at sentence level, in supplementary materials. The performance is higher for recall than for precision. This was designed on purpose, since there is a stronger need to not miss safety signals publication than achieving a higher precision. This optimization towards recall was especially enabled by the additional post processing modules described in Section 3. 3. 4. As far as the total number of collected documents and meaning units is concerned, as displayed in Figure  5 , LiSA is able to perform a very imbalanced prediction task with a high precision. Indeed with more than 53k documents and 3. 8 millions meaning units to filter, there are only 0. 2% of meaning units that should be considered as relevant, for about 10% of all collected documents.
Document level evaluation of LiSA As mentioned before, evaluating the performance at document level is quite challenging. We can calculate the precision using the benchmark dataset available in appendix. Over the 988 documents contained in the benchmark dataset, we found a precision of 78. 5%. Meanwhile, we are not in capacity to provide a good estimation of LiSA document recall. For that purpose, we should be able to measure to which extent the system is able to avoid missing relevant articles in the literature, which would require to label a corpus of at least a few thousand documents (which corresponds to about 80 000 sentences in total). This is an extremely time consuming task and is not immune to potential bias during the document selection phase to build the sample corpus.
Comparison with state of the art models The lack of extensive work on seriousness detection of Adverse Drug Reactions in the literature makes the comparison difficult to perform. In addition to that, the only publication  [10]  that tackles the problem does not provide any code implementation. Thus, apart from re-implementing the solution, there is no possibility to compare our algorithm with the one of this publication. on a corpus extracted from Medical Literature, our pipeline reached a higher performance up to 0. 81 in precision and 0. 88 in recall (respectively compared to 0. 83 and 0. 82  [10]. Even if the dataset are not strictly comparable, we can conclude that our pipeline reached a state of the art performance on the specific task of seriousness classification. To perform a wider comparison with the existing models, additional work is needed and will be described in a forthcoming publication.