dataesr/scientific-paragraphs-categorization huggingface dataset:
The mutation data-update pipeline will continue to be run in quality control mode with every release of IntAct to ensure the mutations data set is kept entirely up-to-date with UniProtKB. Data set statistics. The full IMEx mutations data set contains 27,868 fully annotated events in which a sequence change has been experimentally tested in an interaction experiment. All this information has been manually curated, representing over 33,000 person-hours' worth of biocurators' work, and it is continuously growing with on-going IMEx curation activities. The 4353 proteins annotated come from 297 different species, with over 60% of the events annotated in human proteins and roughly 90% annotated in seven main model organisms (see Table  2 ). In total, 13,926 interaction evidences are annotated with differentially reported effects, using the PSI-MI controlled vocabulary.
Protein-protein interaction (PPI) experiments reporting this type of data have been steadily increasing in the last 20 years, with over 4100 publications containing data pertaining to mutated protein sequences curated by the IMEx Consortium. However, the fraction of PPIs in which a mutated version of a protein has been reported remains relatively low (Fig. 1b ). The majority of the interactions where a mutated protein was involved were detected using either affinity chromatography-related methods (such as co-immunoprecipitations or pull-downs) or by complementation assays based on transcriptional reporters, mainly variations of the yeast two-hybrid method (see Fig. 1c ). Most of our data set comes from the curation of small-scale papers each reporting only a few mutations (Fig. 1d ). Around 99% of the publications (4173) contain less than 100 mutation annotations and represent 80% of the annotations (22,218).
Only 8 publications contain over 100 annotations, with one of them describing over 4000 events, a study in which the authors systematically tested large numbers of variants and their effect on interactions  5. Recording large-scale data sets such as this one has been enabled by the development of the flexible PSI-MI XML3. 0 format cited above. Currently, the only resources that represent the impact of amino acid substitutions on binding events are the SKEMPI 2. 0 database  14  , UniProtKB and IMEx Consortium member databases through IntAct (see Table  3  for a detailed comparison). Of these resources, IMEx is the biggest and the only one that can provide easily accessible, systematically described, up-to-date annotations. UniProtKB mutagenesis annotations record whether a change in sequence affects an interaction, but the experimental context is not captured and the effects are described in a semi free-text field that is difficult to parse.
SKEMPI offers a detailed overview of sequence change effects on binding derived from in vitro experiments, recording changes in affinity and other kinetic parameters for protein complexes with available structural data in the Protein Data Bank (PDB)  15. Only very specific interaction detection methods, using purified proteins, are considered, which limits its scope. There is limited overlap between these resources, with only 4 publications and 44 reported sequence changes found in all three. We manually assessed the consistency of the annotations for these 4 publications and found a number of inconsistencies caused by different curation practices. Most notably, SKEMPI 2. 0 reports mutation coordinates using the chains as annotated in PDB, which often differs from the actual amino acid positions as reported in UniProt entries, used by IMEx and UniProt.
We also found cases where mutations with little or no effect over interaction outcome were not reported in the IMEx data set, a result of early curation guidelines, which did not require the annotation of a mutation unless an effect was clearly shown. Additionally, UniProt and SKEMPI 2. 0 report mutations for which the actual evidence was originally generated in a publication referenced in the one attached in the record. According to IMEx guidelines, all mutation annotations must be referred to the original publication. Finally, some annotations were just missing in one or more of the resources for no obvious reason, although we could attribute some cases to non-parseable UniProt records. Full details of our manual assessment can be found in Supplementary Data 1. The IMEx Consortium is currently formed by 11 groups, each one with their own area of interest, that have agreed to use the same curation standards and data representation download formats.
All members of the consortium  [16] [17] [18] [19] [20] [21] [22] [23]  use the curation platform provided by the IntAct team at EMBL-EBI. Figure  1e  shows the number of events annotated by each data resource. Large databases such as IntAct, DIP and MINT, with an exclusive focus on interaction data curation, have produced the majority of the annotations, but a sizeable part of the data set has been entered by other, domain-specific, members of the Consortium. According to the IMEx schema and curation policy, interaction evidence, rather than interacting pairs of molecules, is the focus of the data representation. This results in the curation of multiple 'Conflict' cases are those in which the effects reported are antagonistic (e. g. 'disrupting' vs 'increasing'). 'Mild conflict' cases are those in which the mutation is sometimes reported as having some effect vs others in which there is no detectable effect distinct pieces of evidence describing the same interacting pairs and offers a way to weight how well characterized is a given interacting group of molecules.
It also enables us to capture separate experiments where different sequence variants are tested for their effect on an interaction. Most of the proteins in the data set have a low number of associated mutations, with most proteins having less than 15 annotations (Supplementary Figure  2a ) and 5 or less sequence changes (Supplementary Figure  2b ). There is a greater depth of information available for human proteins, since the relative amount of human data vs other species increases with the number of annotations per protein. The IMEx evidence-centric curation model also makes it possible to check whether the same mutation has been tested on identical interacting molecules using different interaction detection methodologies (or by different research groups) and whether the outcome of the mutations has been consistent in all these experiments.
Alanine is by far the most frequently used residue for replacement (Fig. 2b ), which is probably reflective of the widespread use of alanine scanning  25  to identify residues critical for binding to other molecules, either because they are found on the interacting interface or at an allosteric binding site. When we checked the relative proportion of the different mutation effects per replacing residue (Fig. 2c , Supplementary Figure  3b ), alanine replacements mostly associate with deleterious effect on interactions. The dominance of deleterious effects most probably reflects the authors of the original study using alanine scanning to locate binding-related residues. Genomic variation and the IMEx mutations data set. In this era of deep-sequencing genomics, there is a wealth of data concerning nonsynonymous genomic variants.
As discussed before, the motivation behind the design of these experiments varies, and only a fraction were specifically designed to systematically test known variants vs reference (wild type) versions of the participant proteins  5, 26. Hence, we decided to explore how much currently available information for natural or disease-related variation can be linked to the data set. Because of the strong predominance of human data both in IMEx mutations and in variation data sets, we decided to focus on human proteins only. We used the EMBL-EBI Proteins API  27  to access variation data both manually annotated by and mapped to UniProtKB from large-scale sequencing studies such as the 1000 Genomes  28  , ExAC  29  and COSMIC  30  projects. We queried 8820 sequence changes in 1990 human proteins, corresponding to 16,765 IMEx mutation annotations (see Table  4  and Fig.
3 ). 29% (4804) of the mutation annotations (Fig. 3a ) and 12% (1073) of the sequence changes (Fig. 3b ) were fully mapped to natural variants. We also checked cases in which there is a variant described in the same position as a mutation reported in the IMEx data set, but the amino acid change is different in the two data sets (positional matches), and also those where mutations span more than one residue and only some of the residue changes or positions are matched in UniProtKB (partial matches). Sixteen percent (2671) of the mutation annotations (Fig. 3a ) and 16% (1415) of the sequence changes (Fig. 3b ) are positional or partial matches. The biological significance of positional and partial mappings does not go beyond stating that the region or position in question is important for interaction and is variable.
However, we believe this information might be useful for researchers interested in exploring specific regions in more detail. We also checked how many of the mapped variant annotations have been linked to disease according to UniProtKB. Disease associations were complemented with data from the DisGeNET   32  and imports of cross-referenced data from ClinVar  33  via Ensembl 34 , while DisGeNET also includes variants from the GWAS Catalog  35  , and from text-mining the scientific literature. Figure  3c  shows the distribution of the variant associations for different disease classes using the Medical Subject Headings (MeSH)  36  classification. A detailed list of these mutation annotation-disease associations can be found in Supplementary Data 3. We then checked if the proportion of disease-related annotations in IMEx varies depending on the reported effect on interaction.
Computational predictions and literature curation. There is currently a variety of computational tools used to annotate variation data sets  38. These tools can report the effect of variation on protein function, folding or binding, usually based almost exclusively on sequence or structural data, or can also report genome-derived parameters such as allele frequencies or conservation scores. We wanted to study how variation annotations provided by these tools align with experimental effect over interaction as reported in the literature. For this purpose, we used mutfunc (www. mutfunc. com)  39 , a database reporting the effect of almost any possible mutation on protein stability, interaction interfaces, post-translational    modifications, protein translation, conserved regions, and regulatory regions.
In this case, mutations predicted to be deleterious by PolyPhen2 were significantly enriched in the 'decreasing' and 'disrupting' effect groups vs those with 'no effect' (Fig. 4f ). The interaction-perturbing effects reported in the IMEx data set can be caused by modifying overall protein structure or by alteration of binding interfaces. We can determine if the mutations reported fall within sequence regions associated with binding using both computational predictions and literaturereported experimental data. We obtained predicted interfaces, based on available structural data, from Interactome3D 41. Literature-curated interfaces were inferred from IMEx records that contained participant features of the 'binding-associated region' (MI:0117) branch. These represent experiments where the authors have tested fragment constructs in an attempt to find sequence regions that are critical for binding, although they may not necessarily represent the actual binding surface.
As seen in Fig. 4g, h , most of the mutations fall within predicted or curated interfaces. The proportion of mutations having an effect over the interaction seems to be higher in binding interfaces, both predicted and inferred from IMEx curation. Disease-associated variants seem to show the same pattern (Supplementary Figure  4a, b ). Thus, the majority of the variants reported to have effects on protein interactions (68%) can be linked to perturbations inside binding regions, with a smaller proportion of variants (32%) potentially representing systemic or allosteric effects influencing interactions. Phosphorylation and mutations in interactions. Posttranslational modifications (PTMs) can be regarded as chemical switches with the potential to influence protein interactions  44. We explored whether mutations with reported effects in our data set are enriched in known sites of protein modification, focusing on phosphorylation in human proteins as the best characterized PTM data set.
5c ). Literature bias in the IMEx mutations data set. IMEx databases have a wide scope when selecting publications for curation and it is reasonable to assume that the proteins in this data set are representative of the interaction data that has been explored in the literature. Socially driven, literature bias is a well-known phenomenon previously reported for literature-curated data sets  26, 46  , so we decided to explore to what extent it affects the data set. First, we checked whether the number of annotations and variants found in the data set and the number of publications in which the affected protein is reported are correlated. As seen in Fig. 6a, b , the data set contains examples of both heavily researched proteins with a low number of annotations and variants and vice versa. If we fit linear models between the number of annotations/ variants and number of publications in which a protein is reported we find a slight positive correlation, especially in the case of disease-related variants.
This observation is compatible with socially-induced bias, with known disease-related proteins and variants being more often reported in the literature. Then we set out to find if the proteins represented in the mutations data set are involved in distinctive pathways vs all proteins for which IMEx has interaction information. To avoid database-specific biases we performed annotation enrichment analysis using PathDIP (http://ophid. utoronto. ca/pathDIP), an analysis tool that integrates information from 20 source databases  47. Human proteins were divided in different sets depending on the effect reported for their mutations and their pathway annotation enrichment was calculated using all the human proteins in IMEx as background. Pathways obtained from these sets have substantial overlap (Fig.
Discussion Here we present a unique resource containing experimental, publicly available information about the impact of sequence changes on specific protein-protein interaction outcomes. This is a direct result of the IMEx Consortium full-detail curation policies and represents an example of how expert curation, resulting in structured and standardized representations, is required in order to make the most of published experimental results. In comparison to similar, pre-existing data sets recording variation influence over interactions, this resource represents a leap forward in depth, size and scope (Table  3 ). A previous, relatively small study  48  reported a curated list of about 100 mutations influencing interactions. Despite obvious limitations due to its size, this was used as benchmark in a study investigating the link between disease-related variation and interaction interfaces  1  , showing the applied potential of this type of data.
The curation infrastructure and practices of the IMEx Consortium will enable the capture of data from a growing number of deep-mutagenesis interaction studies, where hundreds if not thousands of single amino acid changes over the whole length of a protein sequence are explored for their influence on interactions  49. New studies are appearing in which the complex relationship between genetic and protein interactions is explored  50. These epistatic relations can potentially be represented in the PSI-MI data model and the curators in the IMEx Consortium are exploring ways to enable their annotation. We have also acknowledged the social biases inherent to any literature-based resource in our data set, although it is difficult to ascertain its extent. Alanine scanning features prominently as a commonly used technique (Fig.
Our current data set already provides interaction effects for over 10 times more individual variants and is not limited to structural data. The wide scope of experimental setups represented (Fig. 1c ) allows the capture of effects on proteins and protein regions that might be intrinsically nonstructured  53. We show that the data set gives a currently unparalleled and representative overview about which residues are key for protein interactions, with the results being in good accordance with commonly used variant annotators (Fig. 4 ). IMEx curation practices originally did not enforce capturing sequence changes that had no effect over interaction outcome, but as a result of consultations with tool developers and data users this policy has been amended and the data set now features a growing number of mutations with no effect that can be used as a training negative set for the development of computational annotation tools.
Methods Source data. All analysis was performed using the September 2017 version of the IMEx mutations data set, which can be directly downloaded from ftp://ftp. ebi. ac. uk/pub/databases/intact/2017-09-02/various/mutations. tsv. Software and packages used. The quality control pipeline for mutation annotations was developed and integrated within the production code used in the IntAct database. The code is written in Java and makes use of the Hibernate and Spring frameworks for interaction with the core SQL database and application implementation. Specific implementation details are available upon request. Statistical analysis, plots, mutation re-annotation checks and mappings were performed using the R programming language  54  through the RStudio programming suite  55. The following R packages were used in the study: data.
table, dplyr, ggplot2, ggpubr, gridExtra, gsubfn, httr, jsonlite, plyr, RCurl, reshape2, scales, seqinr, splitstackshape, XML, Biostrings, and biomaRt. Curation practices. Data has been produced through manual literature curation following the IMEx Consortium curation guidelines  11  , which can be explored in detail on the Consortium's website: www. imexconsortium. org/curation. Briefly, every publication reviewed was curated for the entirety of the interaction data it Confidence Interval (CI) is shown as greyed areas over the regression lines; c Overlap of significantly enriched pathways (q < 0. 01) across different sets of proteins and word enrichment analysis (using Wordle on enriched pathway names) for the overlapping set (any mutational effect), the set of proteins annotated with no effect and the remaining proteins in IMEx (non-mutated).
0 and the 'resulting sequence' field in the IMEx schemas to capture amino acid change in participant features of the type 'mutation (MI:0118)' and children, it was necessary to populate the field with legacy data from the participant feature short label. This free-text, manually-entered field was prone to contain typographical errors and was difficult to keep updated. Curators used a set of simple rules to depict amino acid substitution, deletions and insertions. As a first step towards populating the 'resulting sequence' field, we wrote ad hoc parsing scripts to evaluate and extract the information stored in the short labels. Several rounds of corrections took place until the data set got to its current state. Of the 27,868 records of the data set, 20,161 had to be corrected, with around 2000 of them manually corrected.
There are still about 2500 records for which no fix was possible without fully amending the original entry. These have been left out of the data set until being revisited by an IMEx curator in due course. An automated quality control pipeline has been put in place to handle newly-created entries and future changes in UniProtKB (details in Supplementary Information section 'Initial re-curation of mutation data in IMEx'). Finally, we have also adapted the participant feature short labels to the Human Genome Variation Society (HGVS) recommendations for variant annotation  56  , which can be accessed at http://varnomen. hgvs. org/recommendations/ protein/. IMEx, UniProt and SKEMPI 2. 0 mutation annotations comparison. The latest version of the SKEMPI 2. 0 database was downloaded in July 2018 from https://life.
bsc. es/pid/skempi2/database/download/. The entries were parsed and the PDB REST API was used to map the proteins in the PDB records to UniProt accession numbers in order to enable comparison. Chain information was used to ascertain the identity of the individual proteins. Cases where more than one protein was reported by chain, as well as unmapped entries, were discarded from further analysis. Finally, effect types were classified in two categories: 'loss' and 'gain' of binding function. In order to classify the annotation in one of these two categories, the affinity values for the mutant version of the sequence were normalized in relation with the wild type form. Fold changes over and under 50% with respect of the wild type were annotated as 'gain' or 'loss', respectively. Smaller fold changes were qualified as 'no effect'.
UniProt mutagenesis annotations were obtained on 29 July 2018 parsing the UniProt webservice XML output using a Python script kindly provided by Luz García-Alonso. Since interacting partners are only identified with gene symbols in UniProt annotations, their precise identity could not be ascertained for a number of records, which were discarded for further analysis. The effects reported in the annotations were simplified and qualified as simple 'loss' and 'gain' of binding function. IMEx mutation types were simplified accordingly to allow consistent comparison of mutation effects among data sets. Assessment of conflicting mutation annotations. A mutation was defined as 'conflicting' when a sequence variant, tested against the same interacting partners, was annotated for effects that are directly antagonistic (e.
g. 'disrupting' vs 'increasing'). Different proteoforms of the same reference protein were considered as different interacting partners (e. g. a phosphorylated protein was considered different from its non-phosphorylated form). When different effects were reported but were not directly antagonistic the mutation was considered to be 'consistent' if all effects went in the same direction (e. g. 'disrupting' and 'decreasing') or 'mild conflict' if most went in the same direction and the rest were annotated as having 'no effect' (e. g. 'decreasing' and 'no effect'). Mapping IMEx mutations to UniProtKB and the genome. UniProtKB accessions for human proteins were extracted from the IMEx mutations data set, retaining isoform identifiers, and used to query the EMBL-EBI Protein API  27. The API's 'variation' method was used to extract large-scale variation annotation from UniProtKB, regardless of its origin.
In the cases in which cross-references to MeSH were not available, we manually assigned a parent, for example, Sporadic Breast Carcinoma was mapped to Breast Carcinoma. This allowed to map 98. 5 and 93% of the diseases in UniProt and DisGeNET, respectively. The following MeSH disease classes were discarded because they were deemed uninformative or not appropriate for the analysis in hand: 'pathological conditions, signs and symptoms', 'occupational diseases' and 'animal diseases'. Predicting impact on protein interaction interfaces. Experimental and homology modelled structures for protein interactions were obtained from the Inter-actome3D database  41. Relative solvent accessibility (RSA) for all residue atoms was computed using NACCESS  57  for proteins individually and in the interaction complex.