dataesr/scientific-paragraphs-categorization huggingface dataset:
It is easy to build a word co-occurrence network from texts by using tools like textexture 1  or GoWvis 2. But they mainly focus on network visualization and cannot handle large corpora such as the English Wikipedia. Our contributions: To address these inconveniences of generating a word co-occurrence network from a large corpus for NLP applications, we propose corpus2graph, an open-source 3  NLP-oriented Python package designed to handle Wikipedia-level large corpora. Corpus2graph supports many language processing configurations, from word preprocessing to sentence analysis, and different ways of defining network edges and edge attributes. By using our node-edge-weight threelevel progressive calculation design, it can quickly build networks for multiple configurations. We are currently using it to experiment with injecting pre-computed word co-occurrence networks into word2vec word embedding computation.
Introduction Genetic and transcriptomic studies are fundamentally reliant on accurate and complete human gene annotation. Gene definitions (namely genic coordinates and the isoforms/exons of which they are composed) are required for the quantification of expression or splicing from RNA-sequencing experiments, interpretation of significant genome-wide association studies (GWAS) signals and variant interpretation from genetic tests. As our understanding of transcriptomic complexity improves it is apparent that existing annotation remains incomplete even amongst known genes. Comparison of different gene annotation databases reveals that over 17,000 Ensembl genes fall into intronic or intergenic regions according to the AceView database and predictably, the choice of reference annotation greatly influences the output of variant interpretation software such as VEP and ANNOVAR 1,2.
Results Optimising the annotation-agnostic detection of transcription using known exons Pervasive transcription of the human genome, the presence of pre-mRNA even within polyA-selected RNA-sequencing libraries and variability in read depth complicates the identification of novel exons and transcripts using RNA-sequencing data  8, 9. With this in mind, we identified a set of exons with the most reliable and accurate boundaries (namely all exons from Ensembl v92 that did not overlap with any other exon 10 ) and then used this exon set to calibrate the detection of transcription from 41 GTEx tissues 11. We used the well- established tool, derfinder, to perform this analysis 12. However, we noted that while derfinder enables the detection of continuous blocks of transcribed bases termed expressed regions (ERs) in an annotation-agnostic manner, the mean coverage cut-off (MCC) applied to determine transcribed bases is difficult to define and variability in read depth even across an individual exon can result in false segmentation of blocks of expressed sequence.
Validation of novel transcription across Ensembl versions and within an independent dataset We recognise that a proportion of novel transcription may originate from technical variability or pre-mRNA contamination. Therefore, we assessed the reliability of novel ERs by classifying ERs using different versions of Ensembl and through an independent dataset. Firstly, we measured how many Kb of the transcription we detected would have been classified as novel with respect to Ensembl v87, but was now annotated in Ensembl v92 and found that across all tissues an average of 68Kb (43-127Kb) had changed status. This value was 5. 3x (3. 2-10. 1x) greater in every tissue compared to the Kb of ERs overlapping exons in Ensembl v87 that had become purely intronic or intergenic in Ensembl v92 (Figure  3a ).
While our analysis of novel ERs across different Ensembl versions provided a high level of confidence in the quality of ER calling, it was limited to ERs which had already been incorporated into annotation and did not provide an overall indication of the rate of validation across all ERs. Therefore, we investigated whether our GTEx frontal cortex derived ERs could also be discovered in an independent frontal cortex dataset reported by Labadord and colleagues 15. As expected, ERs which overlapped with annotated exons had near complete validation (>= 89%), but importantly 62% of intergenic and 70% of intronic ERs respectively were also detected in the second independent frontal cortex dataset (Figure  3b ). While this high validation rate implied the majority of all ERs were reliably detected, we investigated whether a subset of ERs supported with evidence of RNA splicing as well as transcription would have even better rates of validation.
7-3. 8Mb of intronic and 0. 5-2. 2Mb of intergenic transcription was detected across all 41 tissues. Thus, in summary, the majority of novel ERs were reliably detected and validated in an independent dataset. Unannotated expressed regions are depleted for genetic variation and some have the potential to be protein coding suggesting they are functionally significant Given recent reports suggesting widespread transcriptional noise and acknowledging that transcription, even when tissue-specific, does not necessarily translate to function we investigated whether novel ERs were likely to be of functional significance using measures of both conservation and genetic constraint  14, 16. The degree to which a base is evolutionarily conserved across species is strongly dependent on its functional importance and accordingly, conservation scores have been used to aid exon identification 17.
This analysis was performed by using the Systematic Target OPportunity assessment by Genetic Association Predictions (STOPGAP) database, which provides an extensive catalogue of human genetic associations mapped to effector gene candidates (see detailed methods)  22. Interestingly, we found that genes associated with neurodegenerative disorders were significantly overrepresented within our re-annotated set (p-value: 0. 004, Supplementary Table  2 ). In particular, important neurodegenerative disease genes such as SNCA, APOE and CLU were amongst those re-annotated, suggesting that despite being extensively studied the annotation of these genes remains incomplete (complete list found in Supplementary Table  3 ). Thus, we demonstrate that incomplete annotation of brain-specific genes may be hindering our understanding of specific cell types and complex neurodegenerative disorders.
Similarly, we detected a brain-specific novel ER in the long intron of the gene SNCA, which encodes alphasynuclein protein implicated in the pathogenesis of Mendelian and complex Parkinson's disease. This ER connected to two flanking protein-coding exons through split reads (Figure  6d ) and appeared to also have coding potential. Interestingly, while the ER sequence is not conserved within mammals (phastcons7 score: 0. 09) or primates (phastcons20 score: 0. 21), it is in the top 19% of most constrained regions in the genome suggesting it is of functional importance specifically in humans. We validated the existence of this ER both in silico and experimentally. The expression of this ER was confirmed in silico using an independent frontal cortex dataset reported by Labadord and colleagues 15.
We experimentally validate the existence of a novel transcript of SNCA containing a potentially protein coding novel ER. Together, this suggests that incomplete annotation may be limiting our understanding of both Mendelian and common complex diseases. We find that the majority of probable novel exons we detect have a restricted expression pattern across tissues. The practical difficulty of accessing the brain reduces the number of available brain-specific datasets and its regional and cellular heterogeneity is one of the factors driving the high number of brain-specific transcripts. Furthermore, since our approach does not depend on conservation across species to annotate novel exons, we are able to identify ERs which are likely to be of human-specific importance 18. In fact, we find that brain-specific ERs have the highest constraint scores, emphasising their specific importance in humans.
Together these factors suggest that the resource we have generated will have the greatest impact on neurogenetic disorders. Finally, we release our results through a dedicated web resource, vizER (http://rytenlab. com/browser/app/vizER), which enables individual genes to be queried for incomplete annotation as well as the download of all novel ER definitions. We believe that vizER will be an important resource for clinical scientists in the diagnosis of Mendelian disorders, neuroscientists studying individual gene structures and functions, and with the emergence of larger long read sequencing data sets will accelerate novel transcript discovery particularly in human brain.
Optimising the detection of transcription Transcription was detected across 41 GTEx tissues using the package derfinder version 1. 14. 0 12. The mean coverage cut-off (MCC), defined as the number of reads supporting each base above which bases were considered to be transcribed, and max region gap (MRG), defined as the maximum number of bases between expressed regions (ERs) below which adjacent ERs will be merged, were optimised. Optimisation was performed using 156,674 non-overlapping exons (defined by Ensembl v92) as the gold standard 10. Exon biotypes of all Ensembl v92 exons were compared to this set of non-overlapping exons to ensure we were not preferentially optimising for one particular biotype (Supplementary figure  6 ). Non-overlapping exons were selected as these definitions would be least likely to be influenced by ambiguous reads.
Validation of detected transcription Transcription was validated across different versions of Ensembl and within an independent dataset. ERs that overlapped purely intronic or intergenic regions according to Ensembl v87, but fell within exons according to v92, were counted as novel transcription that was validated in later versions of Ensembl. Furthermore, ERs overlapping exonic regions in Ensembl v87 now classified as intronic or intergenic in v92 were measured to control for expected corrections in gene definitions. To assess whether the total Kb of validated novel ERs entering v92 annotation was greater than what would be expected by chance, we generated 10,000 random sets of length-matched regions for each tissue that were intronic or intergenic with respect to Ensembl. Using a one sample Wilcoxon test, we compared the total Kb of intronic and intergenic ERs entering annotation to the total Kb distribution of the randomised intronic and intergenic regions, respectively.
1 Validation within an independent dataset was performed using RNA-seq coverage data from 49 control frontal cortex (BA9) samples originally reported by  Labadorf and colleagues (2015)  and available via the recount R package version 1. 4. 6 4,15. ERs derived from the GTEx frontal cortex (BA9) data were re-quantified using this independent frontal cortex dataset and those that had a mean coverage of at least 1. 4 (the optimised MCC for the GTEx frontal cortex data), were counted as novel transcription that was validated.
Checking ER protein coding potential Intronic and intergenic ERs that were intersected by 2 split reads were extracted. The split reads were used to determine the precise boundaries of the ER. The R package Biostrings version 2. 46. 0 was used to extract the DNA sequence corresponding to the ER genetic co-ordinates from the genome build hg38 28. Since the translation frame was ambiguous without knowledge of the other exons that are part of the transcript that included the novel ER, we converted the DNA sequence to amino acid sequence for all three possible frames starting from the first, second or third base. Any ER that had at least 1 frame that did not include a stop codon was considered to be potentially protein coding.
Gene properties influencing re-annotation All Ensembl v92 genes were marked with a 1 or a 0 depending on whether we detected a re-annotation for that gene in the form of an ER connected to the gene using a split read, with 1 representing a detected re-annotation event. Details of gene length, biotype, transcript count and whether the gene overlapped another gene were 1 retrieved from the Ensembl v92 database. Brain-specificity was assigned using the Finucane dataset and selecting the top 10% of brain-specific genes when compared to non-brain tissues 29. Mean gene TPM was calculated by downloading tissue-specific TPM values from the GTEx portal and summarised by calculating the mean across all tissues. The list of OMIM genes (May 2018) was used to assign whether a gene was known to cause disease or not.
Sanger sequencing was performed using the BigDye terminator kit (Applied Biosystems) and sequences were viewed and exported using CodonCode Aligner (V. 8. 0. 2). Sequences were blatted against the human genome (hg38) and alignment visually inspected for confirmation of validation. Expression-weighted cell-type enrichment (EWCE): evaluating enrichment of theta-correlated genes EWCE was used to determine whether brain-specific genes (both re-annotated and not re-annotated) have higher expression within particular cell types than expected by chance 30. As our input, we used 1) neuronal and glial clusters of the central nervous system (CNS) identified in the Linnarsson single-cell RNA sequencing dataset (amounting to a subset of 114 of the original 265 clusters identified) and 2) lists of genes split by whether or not they were re-annotated, and if re-annotated, by their overlap with Ensembl v92 annotation features (see Supplementary Table  5  for full list of CNS neuronal clusters and genes used) 31.
For each gene in the Linnarsson dataset, we estimated its cell-type specificity (the proportion of a gene's total expression in one cell type compared to all cell types) using the 'generate. celltype. data' function of the EWCE package. EWCE with the target list was run with 100,000 bootstrap replicates, which were sampled from a background list of genes that excluded all genes without a 1:1 mouse:human ortholog. We additionally controlled for transcript length and GC-content biases by selecting bootstrap lists with comparable properties to the target list. We performed the analysis with major cell-type classes classes (e. g. "astrocyte", "microglia", etc. ). Data are displayed as standard deviations from the mean, and any values < 0, which reflect a depletion of expression, are displayed as 0.
Acknowledgements S. G. was supported through the award of an  Alzheimer's Research UK  PhD fellowship. R. H. R. was supported through the  award of a Leonard Wolfson Doctoral Training Fellowship  in Neurodegeneration. J. H. and M. R. were supported by the  UK Medical Research Council (MRC) , with J. H. supported by a grant ( MR/N026004/ ) and M. R. through the award of a  Tenure-track Clinician Scientist Fellowship  ( MR/N008324/1 ). J. H. was also supported by the  UK Dementia Research Institute.
Each scenario computes 50 iterations on matrices with different sizes, and using either only single-precision floating point variables or only double-precision floating point variables. We denote by x b,n the empirical value obtained using b bits and a problem of size n (i. e. the matrix considered will be of size n 3 × n 3 as we consider 3D problems). Then, using Python's lmfit package, we interpolate the data to find good values for a, c and α. We are able to estimate different values of α, all between 0. 20 and 0. 32, using 3 different applications, 2 types of cycles (the classic Vcycle and the Up strategy) and 2 types of relaxation methods (weighted Jacobi and an hybrid method). Each data-fitting was done on either 30 or 40 points. With these values of α, we can estimate the cost of our algorithm in units of time by b 53 α for a cycle (1 unit = 1 V-cycle at double-precision) with b the number of significant bits (i.
This means that even though six members of section Nigri have already been sequenced, ~30% of the predicted gene models in each of the new genomes are not found to encode proteins with homologs in Swiss-Prot. The pan-and core-genome shows genome flexibility. Given the genetic diversity in section Nigri, we were interested in examining the extent of genome diversification. For this analysis, we focused on three conceptual groups of genes: (1) The pan-genome: all genes present in one or more species. (2) The core-genome: genes present in all included species,      1. a, Phylogenetic tree created using RAxML 49 , MAFFT  50  , and gBlocks  51  , based on 2,022 conserved genes. Plate growth pictures are presented for each newly sequenced species. b, Colors indicate whether the organism is from this or another sequencing project.
Articles NaTurE GENETics including paralogs. This set is expected to encode cellular functions needed for all species. (3) Species-unique genes: genes found in only one species in our analysis, with or without paralogs. Included in these, we would expect to find genes involved in environmental adaptation. This group can also include annotation errors. We first identified orthologs and paralogs with a BLASTpbased pipeline using reciprocal hits according to cut-offs specifically selected here for the Aspergillus genus (Methods). Groups of homologous proteins are referred to as families. Figure  2a-c  shows the overall genetic diversity between 38 fungal strains (32 species) from closely related genera (Fig. 2a ), within the Aspergillus genus (36 of the 38 strains; Fig. 2b ), and from section Nigri (32 of the 38 strains; Fig.
Articles NaTurE GENETics genus Aspergillus. However, such a tendency could be also the result of overpredicting genes, considering the low rate of InterPro annotation in the unique genes (Fig. 2e ). The section Nigri core genome contains carbohydrate-active enzymes and secondary metabolism gene clusters. To associate biological functions to the pan-, core-, and unique genomes, and genes exclusive to only members of the black aspergilli, we employed the InterPro database  36. An examination of the coregenome of 38 fungal genomes (Fig. 2a  and Supplementary Fig. 1 ) revealed that only 4. 5% of the genes lack InterPro domains (Supplementary Table  3a ), indicating-as would be expected-that the core-genes across closely related fungal genera include generally known and conserved functions. For the pan-genomes of the 36 Aspergillus species compared with the section Nigri species, the percentages of unknown function are similar (32% compared with 33%; Supplementary Tables 3d and 4a and Supplementary Fig.
2 ), as are the corresponding percentages for the core-genomes (14% compared with 17%, Fig. 2e ; Supplementary Tables  3d  and 4a ). General functions like transporters, regulators, organelle-specific proteins, primary metabolism, and structural domains were found as core features across all 36 aspergilli (Supplementary Table  3f ), which supports the general validity of the method. We expected the section Nigri core-genome (gene families found in all the species of section Nigri but not in any other aspergilli examined) to contain Nigri signature genes, and we found this to be the case. These 1,214 gene families contain 580 InterPro domains conserved to a varying degree, including a many genes involved in the saprotrophic lifestyle and secondary metabolism (Supplementary Table  5 ). It is hypothesized that these genes are defining for the section compared with other aspergilli and will encode functions related to the phenotypes of species in this section.
The prepared SMRTbell template libraries were sequenced on a Pacific Biosciences RS II sequencer using version C2 chemistry and 2 h sequencing movie run times. The three Pacific Biosciences genome datasets were assembled using HGAP3 (see URLs). All genomes were annotated using the JGI annotation pipeline  33. Genome assembly and annotations are available at the JGI fungal genome portal MycoCosm 33 (see URLs) and have been deposited in the DNA Data Bank of Japan (DDBJ)/European Molecular Biology Laboratory (EMBL)/GenBank under the following accession numbers: A. aculeatinus (PSTE00000000), A. brunneoviolaceus (PSTC00000000), A. costaricaensis (PSTH00000000), A. ellipticus (PSSY00000000), A. eucalypticola (MSFU00000000), A. fijiensis (PSTG00000000), A. heteromorphus (MSFL00000000), A. homomorphus (PSTJ00000000), A.
1 mm, 2. 6 μ m) and was run using a linear gradient of wateracetonitrile starting at 10% acetonitrile and increasing to 100% (both containing 50 ppm trifluoroacetic acid) over 8 min, then using 100% acetonitrile for 2 min. The column temperature was 60 °C, the flow rate 0. 8 ml min -1 , and the injection volume was 1 µ l. The ultraviolet spectra 200-640 nm were matched against our internal database 29. Method 2. UHPLC-DAD-HRMS was conducted on a Dionex RSLC UltiMate system linked to a maXis high-definition quadrupole-time-of-flight mass spectrometer (Q-TOF MS) (Bruker Daltonics). Separation was done on a Kinetex C18 column (100 × 2. 1 mm, 2. 6 μ m), with a linear gradient consisting of water and acetonitrile (both buffered with 20 mM formic acid), starting at 10% acetonitrile and increasing to 100% over 10 min, where it was held for 2 min and returned (0.
4 ml min -1 , 40 °C). Injection volume, depending on sample concentration, typically varied between 0. 1 and 1 µ l. Some samples were analyzed in electrospray ionization (ESI) + and some in ESI -full-scan mode, scanning m/z 100-1,250. Data were analyzed by aggressive dereplication  55  using lists of compounds considered to be from black aspergilli only (~350); a list with all Aspergillus compounds (~2,450); and a list of 1,600 reference standards, of which 500 are known to come from Aspergillus. Unknown peaks were matched against Antibase2012 and dereplicated using accurate mass, isotope patterns, adduct patterns, log D, and ultravioletvisible data  58. Method 3. UHPLC-DAD-HRMS was conducted on an Agilent Infinity 1290 UHPLC system coupled to an Agilent 6550 Q-TOF MS. Separation was obtained on an Agilent Poroshell 120 phenyl-hexyl column (2.
1 × 250 mm, 2. 7 µ m) using a linear gradient of water and acetonitrile (both buffered with 20 mM formic acid), progressing from 10% to 100% acetonitrile over 15 min, where it was held for 2 min. The flow was 0. 35 ml min -1 and the temperature 60 °C. Injection volume was between 0. 1 and 1 µ l, depending on the sample concentration. Some samples were analyzed in ESI + and some in ESI -full-scan mode, scanning m/z 100-1,700 and with automatic MS/MS enabled for ion counts above 100,000 and with a quarantine time of 0. 06 min. MS/MS spectra were obtained at 10, 20, and 40 eV (ref. 59 ). Full-scan data were analyzed as above in MassHunter  59. MS/MS data were matched to our internal MS library (~1,700 compounds) of reference standards and tentatively identified compounds  59. Genome annotation and analysis.
NaTurE GENETics constructed on the basis of best bidirectional hits. Two hundred groups with a member from each species were selected, and the sequences of each organism were concatenated into one long protein sequence. Concatenated sequences were aligned using MAFFT (thread 16), and well-aligned regions were extracted using gBlocks (-t = p; -b4 = 5; -b5 = h). Trees were then constructed using multithreaded RAxML, the PROTGAMMAWAG model, and 100 bootstrap replicates. Prediction of SMGCs. For the prediction of SMGCs, we developed a command-line Python script roughly following the SMURF algorithm: According to SMURF the following genes were predicted as a 'backbone' genes: • Genes that have at least three PFAM domains-ketoacyl-synt (PF00109), Ketoacyl-synt_C (PF02801), and Acyl_transf_1 (PF00698)-were predicted as 'PKS' genes.