Package: db-check
Sanity check a FASTA DBFASTA DB are easy to corrupt, and we are in need of a simple tool
to double check the DB has no unforeseen entries.This tool will check for:Duplicate entries with the same IDEntries with different IDs but with overlapping sequences (i.e., 100% overlap, or one sequence completely included in another)Optionally, it will try to parse the meaningful category or label off the FASTA header (e.g., ST allele number) and ensure that distinct labels are given to completely or partially overlapping sequences.The tool outputs:A log tostderrsignalling any potential issues.A report in Markdown straight tostdout. You can then add it to your GitHub, or post it as a GitHub Issue, or push it throughpandocto transform it in to any supported format you like (e.g., PDF for your paper).Optionally, the output files produced byCD-HITWhat does completely and/or partially overlapping sequences mean?Under the hood,db-checkis runningCD-HIT(in particular,cd-hit-est). Here,db-checkrunscd-hit-estto identify any sequences that have 100% identity and 100% coverage (i.e., have all the same bases and are exactly the same length, and thus are identical) OR where one sequence is 100% identical to another, but it is shorter (i.e., one sequence is exactly containted within another). The first case is what we callcompletely overlappingand the second case is what we callpartially overlapping.Obviously, neither case is ideal. The case ofcompletely overlappingmeans your DB has redundant sequences that should be removed. It can be especially problematic if you have two identical sequences but they are identified as distinct categories.The case ofpartially overlappingsequences is less clear cut whether it is an issue or not. We have seen cases where novel MLST alleles were added to a scheme as exact subsets of already established alleles. In those case, it turned out the new alleles were caused by a break in the assembly, and thus wereNOTnovel alleles at all. However, there are cases where partially overlapping sequences might be biologically real (e.g. indicating a real deletion or insertion), and you do wish to keep that in your DB. If that is the case, additional logic will have to be applied to your BLAST filtering steps to make sure you get the right variant.Whatever your case, hopefully, this tool will help you get a quick assessment of your FASTA DB, and be aware of any internal issues before releasing it in to the wild. Or, if you are a user of a FASTA DB, and you are getting funny results, hopefully this tool can help you troubleshoot the DB and pass on the developer any issues they may have with their FASTA DB.DependenciesCD-HITPython >=3.6Optionally,pandoc.InstallationCondaconda -c bioconda cd-hit
pip3 install db-checkBrewbrew install cd-hit
pip3 install db-checkNOTE: I found thatbrew installon my MacOSX Mojave machine failed because the bottled version was compiled with a different set of headers. I fixed it by forcing it to install from source like so:brew uninstall cd-hit
brew install -s cd-hitRunningdb-check <db.fasta> > report.mdRunning an exampledb-check --exampleParsing categories from sequence IDUsing--delimiterand--field:In some cases your category is encoded in the sequence ID with a particular delimiter (e.g.,|or~~), and once the ID is split along this delimiter, the field of interest might be the second (--field 1---1because it is 0-indexed) or, perhaps, the last one (--field -1).For example, let us say you have IDs coded with the following pattern:>seq1~~catA. In this case, you would do the following:db-check --delimiter "~~" --field -1 /path/to/dbUsing--regex:In some cases your category might be encoded in a more convoluted manner, and perhaps aregexexpression with a capture group would work best. The above case could thus be equally parsed with:db-check --regex ".*~~(.*)" /path/to/dbUsing acallbackfunctionImplementation pendingCommand-line optionsUsage: db-check [OPTIONS] FASTA

  Check a FASTA DB for potential issues.

Options:
  -d, --delimiter TEXT   When parsing a category from seqid, split on this
                         delimiter (use -1 for last element, -2 for second to
                         last, etc.).
  -f, --field INTEGER    When parsing a category from seqid using a delimiter,
                         keep this field number (0-index).
  -r, --regex TEXT       When parsing a category from seqid extract using this
                         regex.
  -a, --author TEXT      Who is running the check. (default: $USER)
  -n, --db_name TEXT     Name of the Database. (default: filename)
  -t, --threads INTEGER  How many threads to give CD-HIT (default: 1)
  -p, --prefix TEXT      Prefix of output files from CD-HIT (default: cdhit)
  -k, --keep_files       Whether to keep CD-HIT output files (default: False)
  --example              Run an example set
  --version              Show the version and exit.
  -h, --help             Show this message and exit.Examples usingpandocto convert the output to other formatsConvert to HTMLdb-check --example | pandoc --from markdown --to html5 > report.htmlConvert to PDFdb-check --example | pandoc --from markdown --to latex -o report.pdfConver to plain textdb-check --example | pandoc --from markdown --to plain > report.txtTODOadd conda recipeadd brew recipeadd travisCI recipe