Package: perf-ssr
PERFIntroductionPERF is a Python package developed for fast and accurate identification of microsatellites from DNA sequences. Microsatellites or Simple Sequence Repeats (SSRs) are short tandem repeats of 1-6nt motifs. They are present in all genomes, and have a wide range of uses and functional roles. The existing tools for SSR identification have one or more caveats in terms of speed, comprehensiveness, accuracy, ease-of-use, flexibility and memory usage. PERF was designed to address all these problems.PERF is a recursive acronym that stands for "PERF is an Exhaustive Repeat Finder". It is compatible with both Python 2 (tested on Python 2.7) and 3 (tested on Python 3.5). Its key features are:Fast run time. As an example, identification of all SSRs from the entire human genome takes less than 7 minutes. The speed can be further improved ~3 to 4 fold usingPyPy(human genome finishes in less than 2 minutes using PyPy v5.8.0)Linear time and space complexity (O(n))Identifies perfect SSRs100% accurate and comprehensive - Does not miss any repeats or does not pick any incorrect onesEasy to use - The only required argument is the input DNA sequence in FASTA formatFlexible - Most of the parameters are customizable by the user at runtimeRepeat cutoffs can be specified either in terms of the total repeat length or in terms of number of repeating unitsTSV output and HTML report. The default output is an easily parseable and exportable tab-separated format. Optionally, PERF also generates an interactive HTML report that depicts trends in repeat data as concise charts and tablesChange log[0.4.6] - 2021-04-22FixesFixed usage of unit options file input for fastq input.Fixed usage of repeats input file.[0.4.5] - 2020-05-07AddedAnnotation of repeats w.r.t to genomic context using a GFF or GTF file. (option -g).Multi-threading. Parallel identification of repeats in different sequences.Identification of perfect repeats in fastq files.Analysis report for repeats in fastq files.Option to identify atomic repeats.ChangedAnalysis report rebuilt with Semantic ui and Apex Charts.Visualisation of repeat annotation data in analysis report.FixesPython2 compatability fixed.Bug fixes for PyPi compatability.Import error issues.InstallationPERF can be directly installed using pip with the package nameperf_ssr.$pipinstallperf_ssrThis name was chosen for the package so as not to clash with the existingperfpackage.Alternatively, it can be installed from the source code:# Download the git repo$gitclonehttps://github.com/RKMlab/perf.git# Install$cdperf
$pythonsetup.pyinstallBoth of the methods add a console commandPERF, which can be executed from any directory. It can also be used without installation by running thecore.pyfile in thePERFsubfolder:$gitclonehttps://github.com/RKMlab/perf.git
$cdperf/PERF
$pythoncore.py-h# Print the help message of PERF (see below)UsageThe help message and available options can be accessed using$PERF-h# Short option$PERF--help# Long optionwhich gives the following outputusage: core.py [-h] -i <FILE> [-o <FILE>] [--format <STR>] [--version]
               [-rep <FILE>] [-m <INT>] [-M <INT>] [-s <INT>] [-S <FLOAT>]
               [--include-atomic] [-l <INT> | -u INT or FILE] [-a] [--info]
               [-g <FILE>] [--anno-format <STR>] [--gene-key <STR>]
               [--up-promoter <INT>] [--down-promoter <INT>]
               [-f <FILE> | -F <FILE>] [-t <INT>]

Required arguments:
  -i <FILE>, --input <FILE>
                        Input sequence file.

Optional arguments:
  -o <FILE>, --output <FILE>
                        Output file name. Default: Input file name + _perf.tsv
  --format <STR>        Input file format. Default: fasta, Permissible: fasta,
                        fastq
  --version             show program's version number and exit
  -rep <FILE>, --repeats <FILE>
                        File with list of repeats (Not allowed with -m and/or
                        -M)
  -m <INT>, --min-motif-size <INT>
                        Minimum size of a repeat motif in bp (Not allowed with
                        -rep)
  -M <INT>, --max-motif-size <INT>
                        Maximum size of a repeat motif in bp (Not allowed with
                        -rep)
  -s <INT>, --min-seq-length <INT>
                        Minimum size of sequence length for consideration (in
                        bp)
  -S <FLOAT>, --max-seq-length <FLOAT>
                        Maximum size of sequence length for consideration (in
                        bp)
  --include-atomic      An option to include factor atomic repeats for minimum
                        motif sizes greater than 1.
  -l <INT>, --min-length <INT>
                        Minimum length cutoff of repeat
  -u INT or FILE, --min-units INT or FILE
                        Minimum number of repeating units to be considered.
                        Can be an integer or a file specifying cutoffs for
                        different motif sizes.
  -a, --analyse         Generate a summary HTML report.
  --info                Sequence file info recorded in the output.
  -f <FILE>, --filter-seq-ids <FILE>
                        List of sequence ids in fasta file which will be
                        ignored.
  -F <FILE>, --target-seq-ids <FILE>
                        List of sequence ids in fasta file which will be used.
  -t <INT>, --threads <INT>
                        Number of threads to run the process on. Default is 1.

Annotation arguments:
  -g <FILE>, --annotate <FILE>
                        Genic annotation input file for annotation, Both GFF
                        and GTF can be processed. Use --anno-format to specify
                        format.
  --anno-format <STR>   Format of genic annotation file. Valid inputs: GFF,
                        GTF. Default: GFF
  --gene-key <STR>      Attribute key for geneId. The default identifier is
                        "gene". Please check the annotation file and pick a
                        robust gene identifier from the attribute column.
  --up-promoter <INT>   Upstream distance(bp) from TSS to be considered as
                        promoter region. Default 1000
  --down-promoter <INT>
                        Downstream distance(bp) from TSS to be considered as
                        promoter region. Default 1000The details of each option are given below:-i or --inputExpects:FILEDefault:NoneThis is the only required argument for the program. The input file must be a valid FASTA/FASTQ file. PERF usesBiopython'sFASTA parser to read the input fasta files. It accepts both single-line and multi-line sequences. Files with multiple sequences are also valid. To see more details about the FASTA format, seethis page.-o or --outputExpects:STRING (to be used as filename)Default:Input Filename + _perf.tsv (see below)If this option is not provided, the default output filename will be the same as the input filename, with its extension replaced with '_perf.tsv'. For example, if the input filename ismy_seq.fa, the default output filename will bemy_seq_perf.tsv. If the input filename does not have any extension,_perf.tsvwill be appended to the filename. Please note that even in the case of no identified SSRs, the output file is still created (therefore overwriting any previous file of the same name) but with no content in the file.Output for fastaThe output is a tab-delimited file, with one SSR record per line.
The output columns follow theBEDformat. The details of the columns are given below:S.NoColumnDescription1ChromosomeChromosome or Sequence Name as specified by the first word in the FASTA header2Repeat Start0-based start position of SSR in the Chromosome3Repeat StopEnd position of SSR in the Chromosome4Repeat ClassClass of repeat as grouped by their cyclical variations5Repeat LengthTotal length of identified repeat in nt6Repeat StrandStrand of SSR based on their cyclical variation7Motif NumberNumber of times the base motif is repeated8Actual RepeatStarting sequence of the SSR irrespective of Repeat class and strandAn example output showing some of the largest repeats fromDrosophila melanogasteris given belowX       22012826  22014795  ACTGGG  1969    -       328     TCCCAG
2RHet   591337    591966    AATACT  629     -       104     ATTAGT
4       1042143   1042690   AAATAT  547     +       91      AAATAT
2RHet   598244    598789    AATACT  545     -       90      AGTATT
XHet    122       663       AGAT    541     +       135     GATA
X       22422335  22422827  AGAT    492     +       123     GATA
3R      975265    975710    AAAT    445     -       111     TTAT
X       15442288  15442724  ACAGAT  436     +       72      ACAGAT
2L      22086818  22087152  AATACT  334     -       55      TATTAG
YHet    137144    137466    AAGAC   322     -       64      CTTGTOutput for fastqThe output is a tab-delimited file, with data on each repeat class per line.S.NoColumnDescription1Repeat ClassClass of repeat as grouped by their cyclical variations2Number of readsNumber of reads having an instance of the repeat3FrequencyTotal number of instances of the repeat4BasesTotal number of bases covered by the repeat5Repeat reads per million readsNumber of6Instances per million readsStrand of SSR based on their cyclical variation7Repeat Bases per MB of sequenceNumber of times the base motif is repeated8Length distributionStarting sequence of the SSR irrespective of Repeat class and strand9Motif distributionStarting sequence of the SSR irrespective of Repeat class and strand--formatExpects:STRING (specifying format of the file)Default:fastaPERF was originally developed to identify repeats in FASTA files. In version 4.0.0 PERF can identify repeats in FASTQ sequence files as well. The default format the program expects is fasta. Specify input format as 'fasta' for FASTA files and 'fastq' for FASTQ files.-a or --analyzeExpects:NoneDefault:FalseIn addition to the default tab-separated output, PERF can also generate a fully interactive HTML report for easy downstream analysis of the repeat data. The filename will be the same prefix as that of the main output. For example, if the input filename wasmy_seq.fa, the analysis report will bemy_seq_perf.html. An example HTML report, generated from the repeat data ofHomo sapiens(build hg19), can be accessedhere(Right click -> Save As).-l or --min-lengthExpects:INTEGERDefault:12Minimum length cut-off to be considered when finding an SSR. The same cut-off will apply for SSRs of all motif lengths, even if the motif length is not a divisor of this value. In such cases, SSRs that end with a partial motif are also picked if they pass the length cut-off.-u or --min-unitsExpects:INTEGERORFILEDefault:NoneThis option finds SSRs with a minimum number of repeating motifs. The argument accepts either an integer or file. If an integer is specified, the same value is used for all motif lengths. Instead, a specific value for each motif length using a two-column tab-separated file as demonstrated below:$catrepeat_units.txt1102634435262The first column specifies the motif length, and the second column specifies the minimum number of times the motif should be repeated to be considered an SSR. This file can be used to identify repeats with different number of repeating motifs: monomers repeated at least 10 times, dimers repeated at least 6 times etc., using the following command$PERF-imy_seq.fa-m1-M6-urepeat_units.txt-rep or --repeatsExpects:FILEDefault:NonePERF provides an option to limit the search to specific repeat motifs. The repeats of interest should be specified via a file containing 4 tab-separated columns, as shown below:$catmy_repeats.txt
AA1+TA1-
AGAG2+
CTAG2-
GAAG2+
TCAG2-
$PERF-imy_seq.fa-repmy_repeats.txt# Find all A and AG repeats from my_seq.faNote:This option is not allowed when-mor-Moptions are used.-m or --min-motif-sizeExpects:INTEGERDefault:1Minimum length of motifs to be considered. By default, PERF ignores redundant motifs. For example, a stretch of 12 A's is considered a monomer repeat of 12 A's rather than a dimer repeat of 6 AA's. However, this is only true if-mis set to 1. If for example,-mis set to 2, then stretches of 12 A's are reported as dimer AA repeats. If this behavior isn't desired, we suggest using the-repoption (see above) to specify the motifs that should/shouldn't be included.-M or --max-motif-sizeExpects:INTEGERDefault:6Maximum length of motifs to be considered. Setting a large value of-Mhas a non-trivial effect on both the runtime and memory usage of PERF. This is noticeable with-Mvalues above 10.--include-atomicExpects:NoneDefault:FalseSearches for atomic repeats when set toTrue. For example, when minimum motif size is set to 2bp, PERF ignores monomer repeats. When include atomic repeats is set toTrue, PERF identifies AA, CC, GG and TT as dimer repeats.-s or --min-seq-lengthExpects:INTEGERDefault:0Minimum length of the input sequence to be searched for SSRs in bp. All sequences in the input file that are smaller than this length will be ignored.-S or --max-seq-lengthExpects:INTEGERDefault:InfinityMaximum length of the input sequence to be searched for SSRs in bp. All sequences in the input file that are larger than this length will be ignored.-f or --filter-seq-idsExpects:FILEDefault:NoneThis option accepts a file with a list of sequence IDs in the input file that should be ignored. Useful for ignoring contigs, scaffold, or other poor quality sequences. The IDs can be FASTA headers (starting with '>' symbol) or just the names without the '>' symbol.-F or --target-seq-idsExpects:FILEDefault:NoneThis option accepts a file with a list of sequence IDs in the input file that should be analyzed. All other sequences will be ignored. Useful for analyzing specific chromosomes from a large input file. The IDs can be FASTA headers (starting with '>' symbol) or just the names without the '>' symbol.--infoExpects:NoneDefault:FalseThis option when set toTrue, includes information about the input sequence files and repeat summary data in the output file.$tail-5test_input_perf.tsv
gi|514486271|gb|KE346361.1|26677592667775ATC16+5CAT#File_name: test_input.fa#Total_sequences: 2#Total_bases: 6462134#GC: 53.970000-g or --annotateExpects:FILEDefault:NoneInput a genomic feature file to annotate the repeats in the genomic context. PERF accepts both GFF and GTF format genomic feature files. Each repeat is annotated w.r.t the closest gene and classified either as Genic, Exonic, Intronic and Intergenic according to the position of the repeat. Besides this, the repeat is also checked if it falls in the promoter region of the gene. Annotation adds 7 columns to the default perf output which already consist 8 columns.S.NoColumnDescription9Gene nameName of the closest gene10Gene StartStart position of gene in the Chromosome11Gene StopEnd position of gene in the Chromosome12StrandThe strand orientation of the gene13Genomic annotationAnnotation of the repeat w.r.t to the gene. Possible annotations are {Genic, Exonic, Intronic, Intergenic}14Promoter annotationIf repeat falls in the promoter region of the closest gene. The default promoter region is 1Kb upstream and downstream of TSS.15Distance from TSSDistance of the repeat from the TSS of the gene.--anno-formatExpects:STRINGDefault:GFFOption to specify the format of the input genomic feature file. Accepted  inputs are GFF or GTF. More details about the GFF and GTF formats can be foundhere.--gene-keyExpects:STRINGDefault:geneThe attribute key used for the name of the gene in the GFF/GTF file. In the below example GFF file, we have the location of a gene and it's mRNA and exon locations. The last column of the file specifies attributes associated with each feature, like ID, Parent, gene etc. PERF uses on of the attribute to identify the gene and also it's exons. In th below example the key "gene" can be used to identify gene and the exons of the gene as they have the same gene name. Please check your GFF/GTF file for a robust attribute key which can identify all genes and their corresponding exons. We are actively working on better annotation where we can identify genes and their exons based on the ID and Parent.# Sample GFF
NC_004354.4	RefSeq	gene	124370	126714	.	-	.	ID=gene1;Name=CG17636;gbkey=Gene;gene=CG17636;gene_biotype=protein_coding;gene_synonym=DmelCG17636,EG:23E12.1;
NC_004354.4	RefSeq	mRNA	124370	126714	.	-	.	ID=rna1;Parent=gene1;Name=NM_001103384.3;gbkey=mRNA;gene=CG17636;transcript_id=NM_001103384.3
NC_004354.4	RefSeq	exon	126626	126714	.	-	.	ID=id13;Parent=rna1;gbkey=mRNA;gene=CG17636;transcript_id=NM_001103384.3
NC_004354.4	RefSeq	exon	125495	126259	.	-	.	ID=id14;Parent=rna1;gbkey=mRNA;gene=CG17636;transcript_id=NM_001103384.3--up-promoterExpects:INTDefault:1000Upstream distance(bp) from the TSS of the gene to be considered as promoter region. Default 1000.--down-promoterExpects:INTDefault:1000Downstream distance(bp) from the TSS of the gene to be considered as promoter region. Default 1000.--versionPrints the version info of PERF.ExamplesThe following examples assume that the file with input sequence in FASTA format is namedmy_seq.fa.Basic Usage# Find all monomer to hexamer repeats of >=12nt length$PERF-imy_seq.fa# Specify output filename$PERF-imy_seq.fa-oPERF_output.tsv# Specify fastq format$PERF-imy_seq.fastq--formatfastqGenerate Analysis Report# Find all monomer to hexamer repeats of >=12nt length and generate an HTML report$PERF-imy_seq.fa-a# Specify output filename$PERF-imy_seq.fa-oPERF_out.tsv-a# HTML file is called PERF_out.htmlAnnotate Repeats# Find all monomer to hexamer repeats of >=12nt length and annotate them$PERF-imy_seq.fa-gmy_seq.gff# Specify feature file format and set downstream promoter region to 500bp$PERF-imy_seq.fa-gmy_seq.gtf--anno-formatgtf--down-promoter500# HTML file is called PERF_out.htmlSet Cut-off Criteria# Find all monomer to hexamer repeats of >=15nt length$PERF-imy_seq.fa-l15# Find SSRs with at least 6 repeating motifs (for all motif lengths)$PERF-imy_seq.fa-u6Identify Specific Repeats$catmy_repeats.txt
AGAG2+
CTAG2-
GAAG2+
TCAG2-# Find all AG repeats and generate an HTML report$PERF-imy_seq.fa-repmy_repeats.txt-aChange Motif Length Cut-offs# Ignore monomer and dimer repeats, and repeats with <4 repeating units$PERF-imy_seq.fa-m3-u4# Report only tetramer repeats of >=16nt length, and generate HTML report$PERF-imy_seq.fa-m4-M4-l16-aIn all the above examples, the output of PERF is saved tomy_seq_perf.tsvand the HTML report is saved tomy_seq_perf.htmlunless-ois specified.CitationIf you find PERF useful for your research, please cite it as follows:PERF: an exhaustive algorithm for ultra-fast and efficient identification of microsatellites from large DNA sequencesAkshay Kumar Avvaru, Divya Tej Sowpati, Rakesh Kumar MishraBioinformatics, , btx721doi: 10.1093/bioinformatics/btx721ContactFor queries or suggestions, please contact:Divya Tej Sowpati -tej@ccmb.res.inAkshay Kumar Avvaru -avvaru@ccmb.res.in