dataesr/scientific-paragraphs-categorization huggingface dataset:
Data Availability Statement: The CHIMERE regional chemistry transport model  [18, 19]  and its technical documentation can be found here: www. lmd. polytechnique. fr/chimere/ (accessed on 1 June 2019). The ANNI-NH 3 -v3. 0R NH 3 total columns [39] can be found here: https://iasi. aerisdata. fr/nh3/ (accessed on 1 June 2019).
Global genetic structure of the pathogen tracks the historical spread of wheat We assessed the evolutionary trajectory of the pathogen in conjunction with the history of global wheat cultivation (Fig. 1A ). For this, we assembled a worldwide collection of Z. tritici isolates from naturally infected fields (Fig. 1B ). We selected isolates covering most wheat production areas, both in the center of origin of the crop (i. e. , the Fertile Crescent in the Middle-East), and in areas where wheat was introduced during the last millennia (i. e. , Europe and North Africa), or last centuries (i. e. , the Americas and Oceania; Fig. 1C ). We gathered 1109 high-quality, whole-genome, short-read sequencing datasets (Table  S1 ) covering 42 countries and a broad range of climates. Using a joint genotyping approach, we produced raw variant calls for further inspection.
To assess genotyping accuracy, we used eight isolates with replicate sequencing data to analyze discrepancies. We adjusted quality thresholds targeting specifically the type of genotyping errors observed in our data set (Fig. S1 ). The improved filtering yielded 8,406,818 high-confidence short variants (short indels and SNPs). The final variant set included 5,578,488 biallelic SNPs corresponding to 14. 1% of the genome. CC-BY 4. 0 International license perpetuity. It is made available under a preprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in The copyright holder for this this version posted September 2, 2022. ; https://doi. org/10. 1101/2022. 08. 26. 505378 doi: bioRxiv preprint We tested whether global diversity patterns of pathogen populations are likely a consequence of the history of wheat cultivation.
Sample collection, culturing and sequencing Fungal isolates were grown on V8, yeast sucrose broth (YSB) or yeast malt agar (YMA) plates or liquid culture prior to DNA extraction. Complete information about the geographic origin, date of collection, available sequencing datasets and references are given in Table  S1. Lyophilized or frozen fungal tissue was used for DNA extraction with QIAGEN kits (DNeasy Plant Mini Kit, QIAcube HT). Sequencing libraries were prepared from sheared DNA on an Illumina sequencing platform following TruSeq library preparations. For the collection from the Joint Genome Institute (see Table  S1 ), DNA was extracted from single-spore isolates, the fragments were treated with end-repair, A-tailing, and ligation of Illumina-compatible adapters (IDT, Inc) using the KAPA-Illumina library creation kit (KAPA biosystems).
Draft de novo assembly and variant calling procedures We created de novo draft assemblies using the software SPADES v. 3. 14. 1  37  with the "careful" method. We used only the assemblies (filtered to remove any contigs shorter than 1 kb) with fewer than 1600 contigs and a total length between 30 and 4 Mb. We trimmed and filtered the reads with Trimmomatic v. 0. 39, thereby removing adapter sequences, trimming leading and trailing bases with a quality lower than 15 for the resequencing of 2020 and 10 for all previous resequencing, and removing sequences shorter than 50 bp  38. The trimmed reads were mapped to the reference genome of IPO323 20 using bowtie2 v. 2. 4. 1  39. GATK v4. 1. 4. 1 was used for short-variant calling with the commands HaplotypeCaller, CombineGVCFs, and GenotypeGVCFs, setting the ploidy to 1 and the maximum number of alternative alleles to 2  40.
To filter out erroneously called short indels and SNPs, we started with a standard set of hard filters using the GATK quality metrics, for which the thresholds were set based on visualization of the metrics across the called variants. The per-site filters included: FS > 10, MQ < 20, QD < 20, ReadPosRankSum between -2 and 2, MQRankSum between -2 and 2, and BaseQRankSum between -2 and 2. We also included a per-genotype filter, removing any genotype with a depth lower than 3. We further assessed the quality of our SNP calling using 8 isolates which were sequenced two times (including in some cases in different sequencing datasets). We made the assumption that the real variation between these pairs should be close to 0, although we cannot completely exclude the possibility of a small number of mutations happening during culturing or maintaining of these isolates in collection.
We used the differences, i. e. , erroneously called variants, between the resequencing pairs as an estimation of genotyping errors and to identify the causes of genotyping errors. Most of the erroneously called variants remaining after the hard filtering were related to genotypes with near-equal numbers of reads supporting the reference allele and an alternative allele, i. e. , "heterozygous" alleles. Such genotypes were called with a high confidence despite the fact that such a heterozygous-like pattern should be recognized as errors in a haploid organism and could be due to misalignment or repeated sequences in the genomes. We consequently implemented an allelic balance custom script to recognize such positions and to filter out any genotype that had fewer than 90% of reads supporting the called allele.
Population structure and population-level statistics We used a subset of the filtered biallelic SNPs (one SNP every 1 kb, no missing data and a minor allele frequency of 0. 05) to estimate the population structure of our worldwide Z. tritici collection. This was done separately with a principal component analysis (R package SNPRelate  44  ) and with a snmf clustering method from the LEA package  45. The clustering analysis ran for a value of K (i. e. , the number of clusters) ranging from 1 to 15 and with 10 repetitions per K. To identify the best K, we used the entropy method implemented in snmf which evaluates the quality of fit of the model to the data, as well as the smallest cluster size and the number of isolates assigned to any cluster with a coefficient higher than 0. 75 (considered as non-admixed).
For the analyses relying on the comparison of distinct groups, we discretized the populations by using only isolates belonging to one of the populations with a proportion higher than 0. 75 at K=11, the inferred best number of clusters. These genotypes were then used as input for treemix which infers splits between populations and creates a population tree  46. We ensured that the tree shape was consistent regardless of possible migration events by running treemix with a number of possible migration events ranging from 0 to 6. We used the assembled genomes of Z. ardabiliae and Z. passerinii as outgroups to root the tree  47. We used the scikit-allel python package to measure genetic diversity (pi), taking into account only the non-admixed isolates from each cluster, in non-overlapping windows of 1 kb 48.
Transposable elements and repeat-induced point mutations We called the TE insertions with the software ngs-te-mapper2  49. In this process, the sequencing reads are first queried against a library of TE sequences, for which we used the TE consensus sequences obtained from 20 fully assembled genomes of 19 global isolates  50. The "junction reads" that align both on a TE consensus sequence and on the flanking genome are used to determine the site of insertion of reference and non-reference TEs. To take into account any inaccuracies in the detection of the insertion sites, the positions of the insertions were rounded to 100 bp, so that insertions of the same element in a short window were considered to be the same insertion for further analyses. The insertions found in more than 10 samples were used to create a PCA (prcomp function from the stats R package), clustering the isolates based on their shared TE insertions.
We investigated the genomic distribution of variants, in relation to genomics, transcriptomics and epigenomics estimates. We gathered information from several sources and aggregated the data in 10kb windows. We used transcriptomic data produced previously  51  and analyzed to calculate TPM  47  , representing the gene expression during the necrotrophic and the asymptomatic phases of infection for the reference isolate. To include epigenomics data, we used previously published histone mark ChIPseq data  22  and identified the peaks for several histone marks: H3K4me2, H3K27me3, and H3K9me2. We trimmed and mapped the reads using trimmomatic as above and bowtie, and filtered the mapped reads to keep only those aligning with a quality higher than 30 using samtools. The histone mark peaks were called using macs2 (option -no-model)  52  , and only the regions appearing in both repeats were kept (bedtools intersect).
To remove windows with low variant counts due to a low mappability, we used genmap to estimate mappability and removed windows with a value lower than 0. 85 (threshold estimated visually based on a density plot of windows across the genome)  53. To analyze the repeat-induced point mutations in the collection of genomes, we used the same consensus TE sequences  50  as a reference genome for mapping of reads (as single reads) with bowtie2. Using a custom python script based on the biopython library  54  , we also estimated the GC content and the RIP composite index  55  , an estimate created to detect ratios of dimers indicative of mutations typical of the repeat-induced point mutation process (RIP). This was done on the reads aligning to the TE consensus sequences, thus providing an estimation of RIP in all transposable elements as well as per TE consensus.
Online social networking platforms allow people to freely express their ideas, opinions, and emotions negatively or positively. Previous studies have examined sentiments on these platforms to study their behavior in different contexts and purposes. The mechanism of collecting public opinion information has attracted researchers to automatically classify the polarity of public opinions based on the use of concise language in messages, such as tweets, by analyzing social media data. In this paper, we extend the preceding work where an unsupervised approach to automatically detect extreme opinions/posts in social networks is proposed. The performance of the proposed approach is evaluated on five different social network and media datasets. In this work, we use a semi-supervised approach known as BERT to reevaluate the accuracy of our prior approach and the obtained classified dataset.
Contribution In our recent work of  Pais et al. (2020) , an unsupervised approach has been used to detect extreme sentiments for the text obtained from social platforms. In this paper, we propose to use pre-trained  BERT Devlin et al. (2019)  to validate the efficacy of our prior approach for detecting extreme sentiments. BERT is a deep learning  (Goodfellow et al. 2016 ) model based on a semi-supervised approach, pre-trained on Wikipedia and Book Corpus, a dataset containing more than 10,000 books of different genres. In addition, we use Transfer Learning  (Ruder et al. 2019)  which is typically done for tasks where the dataset contains too little data to train a full-scale model from scratch. The model is then fine-tuned, where the basic model is unpacked and re-trained on the new data with a very low learning rate.
This can potentially lead to significant improvements by gradually adapting pre-trained features to the new data. We use BERT to validate the classified dataset acquired using the previously proposed unsupervised approach  (Pais et al. 2020). This work confirms that the previously classified extreme events are indeed extreme events in most cases, hence reinforcing the validity of the approach. The rest of the paper is structured as follows: The stateof-the-art and related works are briefly reviewed in Sect. 2, and the BERT approach is detailed in Sect. 3. The experimental setup is explained in Sect. 4, and the results and related discussions are presented in Sect. 5. Finally, the paper is concluded in Sect. 6 with some future research directions.
For example, the classification is limited to binary class rather than multiclass classification. There is also a lack of automation for streamlining the process of crawling, cleaning, and storing Twitter content. The work in  Kaur et al. (2019)  uses a deep learning approach for automatic detection of extremism. The data collected by the authors are divided into radical, non-radical, and irrelevant through the use of relevant annotators. Word2Vec has been utilized in this work to generate word embeddings from data. Authors have used LSTM to detect extremism and classify the data as radical, not radical, and irrelevant in the context of India. This work uses specialized annotators to label the data and the labeling of text is based on the characteristics specified by the authors like abuse of Indian military personnel, anti-national discourse, endorsing terrorism/terrorists, and inciting others.
2018). Wagh et al. (2018)  designed a general sentiment classification to analyze whether a data label is available or unavailable in the target domain. The study analyzed the public dataset of four million tweets from Stanford University to predict the sentiment polarity in user opinions. SA using Hadoop, which rapidly runs large datasets on a real-time Hadoop cluster, was presented by  Mane et al. (2014). It is a platform designed to solve large, unstructured, and complex big data problems using the divide-and-conquer approach to data processing. The study used a number-based approach to scale statements into several classes that assigned an appropriate range of different sentiments. SENTA  (Bouazizi and Ohtsuki 2017)  is an SA tool that offers many features to the end-user. The authors collect texts from Twitter and use SENTA to perform multi-class SA on the texts.
Most of these approaches are supervised methods; our research focuses on an unsupervised and language-independent methodology to detect extreme sentiments on social media platforms. Authors in  Liang et al. (2022)  proposed a graph convolutional network that is built on SenticNet  (Cambria and Hussain 2015)  to exploit the affective dependencies of the sentence according to the specific aspect which is called Sentic GCN. It uses a novel solution to build graph neural networks via integrating the affective knowledge from SenticNet in order to strengthen the dependency graphs of sentences. This novel affective enhanced graph model considers both the reliance of contextual words and aspect words and the affective information between opinion words. It first make graphs over dependency trees of sentences and then it enhance the word dependencies of each sentence by incorporating affective information into the graph and highlighting the specific aspect via leveraging the connection between aspect words and contextual affective words.
Sentiment-based Lexicons SenticNet 5  (Cambria et al. 2018 ) encodes denotative and connotative information commonly associated with real objects, actions, events, and people. It avoids the indiscriminate use of keywords and word co-occurrences and instead relies on the implicit meaning associated with common sense concepts. Unlike purely syntactic techniques, Sentic-Net 5 can detect subtly expressed emotions by analyzing multi-word utterances that do not explicitly express emotion but are associated with concepts that do. Here are two examples from the SenticNet 5 dataset: favorite scores 0. 87 (positive), worry scores -0. 93 (negative). SenticNet 6  (Cambria et al. 2020  is a new version of SenticNet  (Cambria and Hussain 2015) , which is a semantic resource for concept-level sentiment analysis available for public use.
It utilizes an ensemble of graph mining and multi-dimensional scaling to connect the conceptual and affective difference between word-level natural language data and the concept-level opinions and sentiments expressed by them. SenticNet 6 combines logical reasoning within deep learning architectures and it is created using a method for knowledge representation that is both top-down and bottom-up. The top-down approach is for the fact that it make use of symbolic models (i. e. , logic and semantic networks) to encode meaning while bottom-up approach uses subsymbolic methods (i. e. , biLSTM and BERT) to completely learn syntactic patterns from data. It deconstructs multi-word expressions into primitives and superprimitives, hence eliminating the need to build a lexicon that assigns polarity to thousands of words and multi-word expressions.
Sentiment analysis datasets SA development and tuning require sizeable labeled training datasets, also known as the SA training dataset. The first step in developing the analysis requires an SA dataset with thousands of statements already labeled as positive, negative, or neutral. Finding training data is difficult because a human expert must determine and label the polarity of each statement in the training data. Using already available training data reduces the time and effort required to develop a new dataset. The work  Friedrich et al. (2015)  uses Sentiment140  (Go et al. 2009 ) and Sen-tiStrength  (Thelwall et al. 2012 ) on a prominent representative set of research articles, explicitly applying some techniques to sentiment analysis of articles circulating on Twitter. The dataset consists of two comma-separated values (CSV) files: one for testing and one for training.
Sentiment140 provides a sentiment value of the tweet on a scale of 0-4, where 0 = negative, 2 = neutral, and 4 = positive. The values have been converted into three sentiment categories: positive, negative, and neutral for better comparison. In our work, we chose the test file for the evaluation of our system. Authors in  Vadicamo et al. (2017)  use Twitter for Sentiment Analysis (T4SA)  (Vadicamo et al. 2017 ) visual dataset, which contains text and multimedia data to examine user sentiment. The authors collected Twitter data via a continuous tracker for six months and used it for a visual assessment of SA. The study of  Smeureanu and Bucur (2012) , which aims to detect users' opinions on movie reviews using the RT-polarity (Pang and Lee (2005 dataset, classified 2000 comments into two different categories.
These two corpora will help us understand whether our approach works well in extremist religious discourse (e. g. , jihadist) and general Islamic discourse. Although a large number of approaches exist and few studies have offered an explicit comparison between SA techniques, the work of  Gonçalves et al. (2013)  shows comparisons of eight popular SA methods in terms of coverage and agreement. Ribeiro et al. (2016)  present a sentence-level comparison of twenty-four popular sentiment analysis methods, based on a benchmark of eighteen labeled datasets. Performance was evaluated on two sentiment classification tasks: negative vs positive and three classes, namely negative, neutral and positive. However, these studies never compare the effectiveness of sentiment analysis methods or sentiment lexicons on the specific task of identifying extreme sentiments, i.
ExtremeSentiLex In our previous work  Pais et al. (2020) , an unsupervised approach for automatic detection of people's extreme sentiments on social networks was proposed. The approach was based on two steps: (1) extreme sentiment generator (ESG)-we automatically build a standard lexicon consisting of extreme positive and negative sentiment terms, and extend that same lexicon with a method based on word embeddings; (2) extreme sentiment classifier (ESC)-to validate the lexicon, using an unsupervised approach for automatic detection of extreme sentiments. We further evaluated our performance on five different social networks and media datasets (Sect. 2. 4). We designed and developed a prototype system composed of two components, i. e. , ESG and ESC. ESG, based on statistical methods, is applied on SentiWordNet 3.
0 and SenticNet 5 to generate a standard lexical resource known as ExtremeSentiLex that contains only extreme positive and negative terms. Additionally, we extend this new lexicon with new terms through the word embedding method  (Mikolov et al. (2013, in order  to study the behavior of our tools when tested with more terms. Antiextremism agencies can also use these lexical resources to find extreme opinion(s) on social networks to counter violent extremism. We embed the lexicons in the ESC and run them on the compilation of five different datasets, constituted of social network and media posts. The purpose of this experiment is to assess the performance of the proposed tool, and this evaluation will validate the proposed hypothesis that the ESC finds posts with extremely negative and positive sentiments in these datasets.
To obtain more objective results, we use a confusion matrix to calculate recall, precision, F1-score, and accuracy to check the performance of the ESC. In the preceding work, we presented and discussed the initial results of each dataset individually in the tables. The arrangements for these tables are different, according to each dataset itself from the original settings. In our case, P-positive, N-negative and neutral are the original polarity of the posts, EP are posts classified as extreme positive, EN means posts classified as extreme negative, and E+INC are posts classified as non-extreme or inconclusive. We concluded that the extended lexicon detects more extreme posts. There is an approximately 2-5% increase in each category for RT-polarity, Sentiment140, TurntoIslam, and Ansar1 datasets.
BERT takes an input of a sequence of up to 512 tokens and outputs the sequence representation. The sequence has one or two segments, where the first token of the sequence is always [CLS] and contains the particular classification embedding, and another special token  [SEP]  is used to separate the segments. BERT picks the final hidden state h of the first token [CLS] for text classification tasks to represent the complete sequence. In order to get the predicted probabilities from the trained model, a softmax classifier is added to the top of the BERT model. Firstly, the dataset is vectorized for feeding it to the classifier since it is originally in text format. Different models are available for vectorizing text but BERT learns contextual-embedding rather than learning context-free, such as in the case of Word2Vec.
Experimental setup In this experiment, BERT is used which is a transformerbased machine learning technique for NLP. The datasets classified in previous work  (Pais et al. 2020 ) are used for training, while the target variable is polarity, a class created by using the ExtremeSentiLex. The experiment is conducted on all five datasets, and at the end, all five datasets are combined to check the overall performance of the proposed method and BERT. This comes in handy as the type of data can be a primary driver in determining the classification. So, the mixing of diverse data challenges the model and allows space for better insights.
Text split for longer text BERT has a limitation of 512 max length for input characters. It means that the input data cannot be longer than 512 for training. Extreme Ansar1 contains 15325 characters long input feature, and extreme TurntoIslam has a maximum text length of 10034 characters. These data cannot be used directly as input for the training BERT model. The simple and rough way is to truncate directly, take the initial part up to 512 characters max, and discard the rest. Although this simple naive method is effective in many cases, a complex method tackles this issue based on the HIERARCHICAL (cascade) idea, which divides the longer text into smaller chunks and feeds them into the base model  (Pappagari et al. 2019). Any text larger than 450 characters is divided into 500 characters with an overlap length of 50 words.