DOI: 10.5281/zenodo.10467208
title: Benchmarking computational tools for de novo motif discovery (MEP-Bench datasets)

description: Supplementary material for the publication. Includes 3 files:



Supplementary Table S2 [TAR.GZ]: Is a compilation of all results obtained from the benchmarking of 5 motif discovery tools and 7 alignment tools run over MEP-Bench, a set of over 600.000 datasets generated from the ELM database.

MEP-Bench - Supplementary Data SD1 [TAR.GZ]: Are all the datasets generated for the publication that signify a resource for the community for benchmarking future tools.

README [txt]: Contains detailed information on the structure of the Supplementary Table S2 and Supplementary Data SD1 (MEP Bench).


Update: The datasets generated for this work, contained in Supplementary Data SD1, have been named Motif Extraction from Peptides Benchmarking (MEP-Bench) datasets.
description: =========[ file: Supplementary_Table_S2.tsv.tar.gz ]=========Supplementary Table S2 - Benchmarking results. Tools run results for all dataset replicates are shown. For each replicate its ID (form 00 to 99), ELM class, PSS, noise peptides number and percentage and peptide length are shown. For the tools run on each dataset the run time and percentage of correctly aligned motifs are shown together with a regular expression or consensus of the motif found if the tool generated it.
description: =========[ MEP-Bench - file: Supplementary_Data_SD1.tar.gz ]=========Supplementary Data SD1 - Motif Extraction from Peptides Benchmarking (MEP-Bench) datasets. The MEP-Bench datasets are organized into 158 folders, eachc one belonging to a ELM class. Inside each ELM class folder there's a single "testsets" folder that contains all generated dataset files. All files follow the naming rule:[ELM class]-[peptide length as an int followed by "len"]-[PSS as an int followed by "pos"]-[noise peptides number as an int followed by "noise"]-[replica number, from 00 to 99]

Example: The replica 58 for ELM class CLV_C14_Caspase3-7 for 16 amino acid peptides, a positive set size of 15 and noise percentage of 200% (that is 30 peptides) would be:CLV_C14_Caspase3-7-16len-15pos_30noise-58

For each dataset a FASTA and a TSV table file is provided. The table contains the information regarding the motifs present in the sequences. The table columns are:



 Group: Positive or Noise

 One_motif_per_peptide: True or False (it will always be True)

 Randomize_motif_position: True or False (it will always be True)

 Exclude_terminal_motif: True or False (it will always be False)

 ELM_class: The ELM class ID

 ELM_regex: The regular expression for the class in ELM database

 Protein: The UniProt Accession ID of the instance's protein

 ELM_instance: Instancec ID in ELM database

 Peptide_position: Internal, C-terminal or N-terminal

 Peptide_length: Length in amino acids of the peptide

 Peptide: Peptide sequence in 1-letter code

 Nbr_of_motifs: Number of the expected motif for the ELM class in the peptide (it will always be 1)

 Main_instance: Tuple of the main motif in the peptide with the structure (motif sequence [string], motif's fist amino acid position in the peptide [int, with first position in peptide == 0], motif last amino acid position in the peptide [int, with first position in peptide == 0])

 Extra_instances: Instances besides the main one are shown here as tuples (with the same structure as in the previous column) and if more than 1 is available they are separated by semicolons(";"). Because a single motif was neforced for the positive set, this column will always be empty for peptides belonging to the "Positive" group (but "Noise" peptides can have multiple instances of the ELM classes they are coming from)


The peptide names in the FASTA file also display most of this information separated by bars ("|").

Besides the dataset files, for each peptided length value a FASTA file containing 100 pure noise peptides (no peptide containing the expected motif for the ELM class) was also generated to use as background (or negatve control) in case the tool requires that. These files follow the naming rule:[ELM class]-[peptide length as an int followed by "len"]-negative_set.fasta

So for following the previous naming example, the pure noise background file woul be called:CLV_C14_Caspase3-7-16len-negative_set.fasta