dataesr/scientific-paragraphs-categorization huggingface dataset:
To construct the network in a time efficient manner, we employed MMseqs2  (Steinegger and Söding, 2017)  for all-against-all sequence searches (Fig. 1B ). In these searches, two sequences were deemed as adjacent if there was an alignment that covered at least 50% of one of the proteins at an Evalue better than 1x10 -4. After simplification of the resulting graph by considering the 4 nearest neighbours of each node, we obtained a graph with 10'339'158 edges involving 4'270'404 nodes, which include 43% of the dark UniRef50 clusters in the full dataset (Fig. 3 ). The resulting graph (Fig. 3A , https://uniprot3d. org/atlas/AFDB90v4) is composed of 242'876 connected components with at least 2 nodes, with the largest encompassing about 50% of all UniRef50 clusters. Out of these components, 46'318 (19%) have an average brightness content below 5% and are, thus, "fully dark galaxies" in the protein sequence space represented by the network (Fig.
(E) Metabolic labelling assays with wild-type E. coli BW25113 expressing A. tepidum TumE/DUF6516 toxin. While most of the "partner" genes associated with DUF6516 code for "hypothetical proteins" of unknown function, one of the partners in cluster 1 is a remote homolog of RelB, a well-characterised antitoxin  (Gotfredsen and Gerdes, 1998). The bicistronic arrangement is typical for toxin-antitoxin (TA) systems  (Jurėnas et al. , 2022). When active, the TA toxin proteins abolish bacterial growth, and the control of this toxicity is executed by the antitoxin, which, in the case of so-called "type II TA systems", is a protein that acts by forming an inactive complex with the toxin. Structure-based molecular function prediction for DUF6516 partners using DeepFRI  (Gligorijević et al. , 2021)  suggests they may bind DNA (Supplementary file 1), an activity characteristic for diverse antitoxins  (Jurėnas et al.
Semantic diversity of protein names in functionally dark connected components Recently, ProtNLM  (Gane et al. , 2022) , a large language model, was implemented as an approach to automatically name proteins in TrEMBL titled as "Uncharacterised protein". Upon the first release of such predictions in UniProtKB, we investigated how diverse the names predicted for those proteins in dark connected components were, and compared it to those fully bright components. In both cases, we observed that the distributions of name and word diversities (collectively referred to as "semantic diversity") across components were highly skewed towards extremely low values, but the fully dark set was significantly different in both cases from the fully bright set (Kolmogorov-Smirnov p-value = 1x10 - 16 , Fig. 6 ).
Conclusions: Towards large-scale protein function annotation In this work, we carried out a large-scale analysis of the catalogued protein sequence space covered by predicted high quality structural information, as made available through AlphaFold database version 4. Our results indicate that 19% of this space is composed of protein families and superfamilies that are functionally dark and can not, solely on the basis of sequence similarity, be annotated to a known protein family. We also highlight that a large fraction of these can not be named consistently using the most recent deep-learning-based approaches and demonstrate that functional annotation of proteins, even from a purely computational perspective, requires a combination of data sources and approaches. Our findings highlight the power of large-scale similarity networks to accurately annotate protein function.
Data collection We started from the 53'625'855 UniRef50  (Suzek et al. , 2015)  clusters as of August 2022 (UniRef version 2022_03) and the 214'683'829 structural models for most UniProtKB entries available via the AlphaFold database (version 4, AFDBv4). For each Swiss-Prot  (Boutet et al. , 2016 ), TrEMBL (UniProt Consortium, 2023)  and UniParc  (Leinonen et al. , 2004)  entry in each UniRef50 cluster we collected their sequence, taxonomy and functional and structural annotations from UniProt and InterPro  (Paysan-Lafosse et al. , 2023). This includes domains, protein families, predicted transmembrane segments, predicted signal peptides, and predicted disorder and coiled coil regions. Redundant, overlapping domain, families, coiled coil and disorder annotations were continuously merged (Fig.
Large-scale sequence similarity network To model the sequence landscape covered by all UniRef50 clusters with a high-confidence structural model, we built a large-scale sequence similarity network of 6'136'321 UniRef50 clusters with a structural representative with a pLDDT  (Jumper et al. , 2021)  better than 90 (AFDB90 dataset). For that, all-against-all MMseqs2 (Steinegger and Söding, 2017) comparisons were carried out with the sequence representatives of all selected UniRef50 clusters, linking two sequences if they find a match that covers at least 50% of their full length sequences with an E-value better than 10 -4. Each edge was given a weight proportional to the E-value of the match and, to facilitate the handling of the resulting network, a maximum of 4 outbound edges were considered per node.
As the output of the network is 10 floating point numbers between 0 and 1, this could be discretized into 10 bits based on whether the value was greater than or less than 0. 5, resulting in 1024 shape-mers. The training set is created from structures from the CATH database having less than 40% sequence identity (CATH40) that could be assigned to a CATH functional family (FunFam) with an Evalue < 1x10 -6. From these 8'333 structures, US-align  (Zhang et al. , 2022)  was used to align and superpose all pairs within each FunFam cluster and 3x8'333 randomly chosen pairs across clusters. Aligned pairs of residues from two proteins belonging to the same FunFam with an alignment TM-score > 0. 8 were considered as positive pairs. Aligned or random pairs of residues from two proteins belonging to different CATH superfamilies, with an alignment TM-score <0.
6 were considered as negative pairs. In addition, using all 31,883 CATH40 proteins, we sampled up to 50 pairs of central residues from each protein, where positive pairs had <2 sequence distance and negative pairs had 5-20 sequence distance. In total, this resulted in 6 million residue pairs for training, of which 42% were positive pairs. This dataset could be used for training and/or refining any kind of residue-level contrastive learning task. Training took 30 mins on 1 RTX-3080TI with the ADAM optimizer, a batch size of 1024, and a learning rate of 10 -3 over 5 epochs. Shape-mers were calculated for all AFDB90 proteins, and all ProteinNet CASP12 proteins in the 100% sequence identity set  (AlQuraishi, 2019). For AlphaFold models, we followed the approach described in the analysis of AFDBv1  (Akdel et al.
, 2022)  to split each protein into segments with Gaussian smoothed plDDT > 70, after first splitting into domains based on a combination of pLDDT and the predicted aligned error (PAE) matrix. Shape-mers were then calculated for each segment in each domain and concatenated. Proteins with less than 20 amino acids were ignored. A shape-mer diversity fraction was defined for each protein as the number of unique shape-mers divided by the total number of shape-mers (which is equal to the protein length if all residues have high pLDDT). Fig. S4  shows an example AlphaFold protein with its 6 most common shape-mers highlighted. We trained a FastText model  (Bojanowski et al. , 2017)  on the shape-mer bit representations from the ProteinNet CASP12 dataset using Gensim  (Rehurek and Sojka, n. d. ) (v4.
2. 0) with a window size of 16 and embedding size of 1024. Fig. S5A  shows the sensitivity of SCOPe family retrieval on the SCOPe40 dataset of 11'211 structures for all-vs-all Smith-Waterman alignment with FastText shapemer similarities used as the score matrix (runtime: 12 mins on 10 threads). Shape-mer FastText alignment scores are compared to three structure aligners, DALI  (Holm, 2020), Foldseek (van Kempen et al. , n. d. ) , and TM-align  (Zhang and Skolnick, 2005) ; one sequence aligner, MMseqs2  (Steinegger and Söding, 2017) ; and 2 other structure alphabet-based structural sequence aligners, 3D-BLAST  (Mavridis and Ritchie, 2010)  and CLE-SW  (Wang and Zheng, 2008) , using the scripts and benchmark data provided in  (van Kempen et al. , n. d. ). The benchmark results demonstrate that the learned structural alphabet and FastText similarities still have discriminative power in distinguishing protein families, despite being much less "local" than approaches such as Foldseek and TM-align which work on individual coordinates of up to 2 residues.
We don't explore further alignment optimization, such as compositional bias correction or penalty optimization to increase sensitivity, as more local structural aligners will still have the advantage of higher resolution alignment. However, for the task at hand, our substructure representations give us a good compromise -a discriminative structural alphabet for representing a protein structure as a structural sequence; and substructure decomposition at the level of whole secondary-structural elements, allowing for a broader exploration of substructure composition across the AlphaFold database. To compare substructure compositions, we obtained protein-level embeddings by averaging across normalised FastText embeddings using the get_sentence_vector function. Fig. S5B  shows the distributions of cosine distances of these embeddings within the same SCOPe family and across SCOPe folds, demonstrating that they have discriminative power in representing substructure compositions.
We trained the Isolation Forest algorithm  (Liu et al. , 2008)  as implemented in scikit-learn v1. 1. 1  (Pedregosa et al. , 2011)  on the ProteinNet CASP12 FastText sentence embeddings with 1% contamination rate, and used this trained model to predict structural outlier scores for proteins in the AFDB90 dataset. Proteins with a negative score are labelled as outliers.
Computational investigation of selected examples We further investigated 4 different types of examples of dark proteins and connected components we encountered in our dataset: (1) a dark connected component whose models are structural inliers and are named consistently, but have no matches in the PDB (component 159), (2) a dark component whose models are structural inliers, are named consistently and have structural homologs in the PDB (component 27), (3) a dark connected component that is named inconsistently (component 3314), and (  4 ) examples of different kinds of structural outliers, including a novel fold (β-flower fold). In all cases, we combined data from the sequence-based network and its functional brightness annotations, as well as from structural searches with Foldseek and the outlier scores.
Structural homologs for selected representatives (those with a length close to the median length in the component) in the PDB or the AFDB90Communities set were searched with Foldseek using the TM-align mode  (van Kempen et al. , n. d. ). Remote sequence homologs were detected for selected representatives by HHPred searches over the PDB, ECOD and Pfam databases through the MPI Bioinformatics toolkit using default settings  (Gabler et al. , 2020; Pereira and Alva, 2021). Further case-by-case analyses were also carried out as described below.
β-flower fold We constructed three new Pfam families to cover the sequence space of β-flower proteins. To do this we selected example proteins with 4,5 and 6-fold rotational symmetry and iteratively searched for homologs using the hmmsearch software of the HMMER package (version 3. 3)  (Eddy, 2011). In general, we used an inclusion threshold of 27 bits, but manually lowered the threshold to identify more homologs or raised it to exclude false matches as identified by AlphaFold2 models. These three families were added to Pfam with accession numbers: PF21784, PF21785 and PF21786 and these families were added to Pfam clan CL0396, which includes the Tubby C-terminal domain. The families were included in the Tubby C-terminal clan considering their significant structural similarity to known structures of Tubby-like proteins, in particular the structure of plant Tubby-like At5g01750 protein (PDBID 1zxu).
Data and code availability All the code to collect and process the annotation data in UniProtKB, UniParc and InterPro, and the pLDDT data from AFDB is available at https://github. com/ProteinUniverseAtlas/dbuilder. All analysis code, data and metadata generated for the current submission is available at https://github. com/ProteinUniverseAtlas/AFDB90v4. In addition, an interactive version of the sequence similarity network, queriable by UniProt ID, protein title, connected component and community ID, is available at https://uniprot3d. org/atlas/AFDB90v4.
Data-driven models of neurons and circuits are important for understanding how the properties of membrane conductances, synapses, dendrites and the anatomical connectivity between neurons generate the complex dynamical behaviors of brain circuits in health and disease. However, the inherent complexity of these biological processes make the construction and reuse of biologically-detailed models challenging. A wide range of tools have been developed to aid their construction and simulation, but differences in design and internal representation act as technical barriers to those who wish to use data-driven models in their research workflows. NeuroML, a model description language for computational neuroscience, was developed to address this fragmentation in modeling tools. Since its inception, NeuroML has evolved into a mature community standard that encompasses a wide range of model types and approaches in computational neuroscience.
, 2020) ,  PyNN (Davison et al. , 2009)  and Neural Open Markup Language (NeuroML)  (Gleeson et al. , 2010) ). These standards have enabled open and interoperable ecosystems of software applications, libraries and databases to emerge, facilitating the sharing of research outputs, an endeavor encouraged by a growing number of funding agencies and scientific journals. The initial version of the NeuroML standard, version 1 (NeuroMLv1), was originally conceived as a model description format  (Goddard et al. , 2001) , and implemented as a three layered, declarative, modular, simulator independent language  (Gleeson et al. , 2010). NeuroMLv1 could describe detailed neuronal morphologies and their biophysical properties as well as specific instantiations of networks. It enabled the archiving of models in a standardized format and addressed the issue of simulator fragmentation by acting as the common language for model exchange between established simulation environments-NEURON  (Hines and Carnevale, 1997; Awile et al.
NeuroMLv2 also moved to Python as its main interface language and provides a comprehensive set of Python libraries to improve usability  (Vella et al. , 2014) , with XML retained as a machine-readable serialization format (i. e. the form in which the model files are saved/shared). Since its release in 2014, the NeuroMLv2 standard, the software ecosystem, and the community have all steadily grown. An open, community based governance structure was put in place-an elected Editorial Board, overseen by an independent Scientific Committee, maintains the standard and core tools. Although the core tools were initially focused on enabling the simulation of models on multiple platforms, they have been expanded to support all stages of the model life cycle (Fig. 1 ). Modelers can use these tools to easily create, inspect and visualize, validate, simulate, fit and optimize, share and disseminate NeuroMLv2 models and outputs  (Billings et al.
, 2014; Cayco-Gajic et al. , 2017; Gurnani and Silver, 2021; Kriener et al. , 2022; Gleeson et al. , 2019). To provide clear, concise, searchable information for both users and developers, the NeuroML documentation has been significantly expanded and re-deployed using the latest modern web technologies  1   Modeling in Biology NEtwork (COMBINE)  (Hucka et al. , 2015) , and which is now sufficiently mature to be incorporated into a wide range of research workflows. In this paper, we provide an overview of the current scope of version 2 of the NeuroML standard, describe the current software ecosystem and community, and outline the extensive resources to assist researchers to incorporate NeuroML into their modeling work. We demonstrate, with examples, that NeuroML supports users in all stages of the model development life cycle (Fig.
NeuroML provides a ready to use set of curated model elements A central aim of the NeuroML initiative is to enable and encourage the use of multi-scale biophysically detailed models of neurons and neuronal circuits in neuroscience research. The initiative takes a number of steps to achieve this aim. NeuroML provides users with a curated library of model elements that form the NeuroML standard 2  (Fig. 2 ). The standard is maintained by the NeuroML Editorial Board which has identified a core set of model types to support, to ensure that a significant proportion of commonly used neurobiological modeling entities can be described with the language. This includes (but is not limited to): active membrane conductances (using Hodgkin-Huxley style  (Hodgkin and Huxley, 1952)  or kinetic scheme based ionic conductances), multiple synapse and plasticity mechanisms, detailed multicompartmental neuron models with morphologies and biophysical properties, abstract point neuron models, and networks of such cells spatially arranged in populations, connected by targeted projections, receiving spiking and current based inputs.
Most users do not need to interact with this level (which is enabled by LEMS), which, among other features, facilitates the automated translation of simulator-independent NeuroML models into simulator-specific code. Thus, modelers can use the standard NeuroML elements to conveniently build simulator-independent models, while also being able to examine and extend the underlying implementations of models. As a simulator independent language, NeuroML also promotes interoperability between different computational modeling tools, and as a result, the standard library is complemented by a large, well maintained ecosystem of software tools that supports all stages of the model life cycle-from creation, analysis, simulation and fitting, to sharing and reuse. Finally, as discussed in later sections, for advanced use cases where the existing NeuroML model building blocks are insufficient, NeuroML also includes a framework for creating and including new model elements.
In practice, many examples of ionic conductances that underlie the electrical behavior of neurons are already available in NeuroMLv2 and can simply be plugged into a cell membrane (Fig. 2 ). Indeed, a model element, once defined in NeuroML, acts as a building block that may be reused any number of times within or across models. Such reuse of model component speeds model construction and prototyping irrespective of the simulation engine used. The defined structure of each model element and the relationships between them inform users of exactly how model elements are to be created and combined. This encourages the construction of well-structured models, reduces errors and redundancy and ensures that FAIR principles are firmly baked into NeuroML models and the ecosystem of tools. As we will see in the following sections, NeuroML's formal structure also enables features such as model validation prior to simulation, translation into simulation specific formats, and the use of NeuroML as a common language of exchange between different tools.
NeuroML supports a large ecosystem of software tools that cover all stages of the model life cycle Model building and the generation of scientific knowledge from simulation and analysis of models is a multi-step, iterative process requiring an array of software tools. NeuroML supports all stages of the model development life cycle (Fig. 1 ), by providing a single model description format that interacts with a myriad of tools throughout the process. Researchers typically assemble ad-hoc sets of scripts, applications, and processes to help them in their investigations. In the absence of standardization, they must work with the specific model formats and APIs that each tool they use requires, and somehow convert model descriptions when using multiple applications in a toolchain. NeuroML addresses this issue by providing a common language for the use and exchange of models and their components between different simulation engines and modeling tools.
The NeuroML ecosystem includes a large collection of software tools, both developed and maintained by the main NeuroML contributors (the "core NeuroML tools") and those external applications which have added NeuroML support (Fig. 3 , Appendix 1 Tables  1  and 2 ). The core NeuroML tools currently include APIs in several programming languages-Python, Java, C++ and MATLAB. These tools provide critical functionality to allow users to interact with NeuroML components and build models. Using these, researchers can build models from scratch, or read, modify, analyze, visualize, and simulate existing NeuroML models on supported simulation platforms. The Table  1  lists a number of interactive, step-by-step guides in the NeuroML documentation, illustrating how software tools can be used to achieve specific tasks across the model development life cycle.
Creating NeuroML models The structured declarative elements of NeuroMLv2, when combined with a procedural scripting language such as Python, provide a powerful and yet intuitive "building block" approach to model construction. For this reason, Python is now the recommended language for interacting with NeuroML (Fig. 4 ), although   1  and 2. XML remains the primary serialization language for the format (i. e. for saving to disk and depositing in model repositories (Fig. 5 )). Python has emerged as a key programming language in science, including many areas of neuroscience  (Muller et al. , 2015). A Python based NeuroML ecosystem ensures that users can take advantage of Python's features, and also use packages from the wider Python ecosystem in their work (e. g. Numpy  (Harris et al. , 2020) , Matplotlib  (Hunter, 2007) ).
pyNeuroML, the Python interface for working with NeuroML, is built on top of the Python NeuroML API, libNeuroML  (Vella et   NeuroML APIs support all these workflows. The Python tools also include many additional higher level utilities to speed up model construction, such as factory functions, type hints, and convenience functions for building complex multicompartmental neuron models (Box 1). For construction of complex 3D circuit models, or for users who are not experienced with Python, there are a number of NeuroML compliant online and standalone applications with graphical user interfaces for model construction. These include NetPyNE's interactive web interface  (Dura-Bernal et al. , 2019)  (which is available on the latest version of OSB 3  ) and neuroConstruct  (Gleeson et al. , 2007)  which can export models directly into NeuroML and LEMS.
b. Example Python usage from neuroml import * # NeuroML API libNeuroML newdoc = NeuroMLDocument(id="new_doc") newcell = IafTauCell(id="cell_0", leak_reversal="-60mV", thresh="0mV", tau="5ms", reset="-70mV") newdoc. add(newcell) network = newdoc. add(Network, id="new_net", validate=False) population = network. add  (Population, id="new_pop", size=10, component=newcell. id)  # Helper method to ensure all parameters # present and appropriate newdoc. validate(recursive=True) c. XML serialization <neuroml id="new_doc"> <iafTauCell id="cell_0" leakReversal="-60mV" thresh="0mV" reset="-70mV" tau="5ms"/> <network id="new_net"> <population id="new_pop" component="cell_0" size="10"/> </network> </neuroml>   The Python API can be used to create models which can include elements built from scratch from the core NeuroML standard, re-use elements from previously created models, or create new components based on novel model definitions expressed in LEMS (red).