Package: itermae
itermaeSee theconcept hereandtutorial here.itermaeis a command-line utility to recognize patterns in input sequences
and generate outputs from groups recognized. Basically, it uses fuzzy regular
expression operations to (primarily) DNA sequence for purposes of DNA
barcode/tag/UMI parsing, sequence and quality -based filtering,
and general output re-arrangment.itermaereads and makes FASTQ, FASTA, text-file, and SAM (tab-delimited)
files usingBiopythonsequence records
to represent slice, and read/output formats.
Pattern matching uses theregexlibrary,
and the tool is designed to function in command-line pipes from tools likeGNUparallelto permit light-weight parallelization.It's usage might look something like this:zcat seq_data.fastqz | itermae --config my_config.yml -v > output.samorzcat seq_data.fastqz \
    | parallel --quote --pipe -l 4 --keep-order -N 10000 \
        itermae --config my_config.yml -v > output.samwith amy_config.ymlfile that may look something like this:matches:
    - use: input
      pattern: NNNNNGTCCTCGAGGTCTCTNNNNNNNNNNNNNNNNNNNNCGTACGCTGCAGGTC
      marking: aaaaaBBBBBBBBBBBBBBBccccccccccccccccccccDDDDDDDDDDDDDDD
      marked_groups:
          a:
              name: sampleIndex
              repeat: 5
          B:
              allowed_errors: 2
          c:
              name: barcode
              repeat_min: 18
              repeat_max: 22
          D:
              allowed_insertions: 1
              allowed_deletions: 2
              allowed_substititions: 2
output_list:
    -   name: 'barcode'
        description: 'description+" sample="+sampleIndex'
        seq: 'barcode'
        filter: 'statistics.median(barcode.quality) >= 35'Availability, installation, 'installation'Options:Use pip to installitermae, sopython3 -m pip install itermaeYou can clone this repo, and install it locally. Dependencies are inrequirements.txt, sopython3 -m pip install -r requirements.txtwill install those.You can useSingularityto pull and run aSingularity image of itermae.py,
where everything is already installed.
This is the recommended usage.This image is built with a few other tools,
like g/mawk, perl, and parallel, to make command line munging easier.Usageitermaeis envisioned to be used in a pipe-line where you just got your
DNA sequencing FASTQ reads back, and you want to parse them.
The recommended interface is the YAML config file, as demonstrated
inthe tutorialand detailed again in theconfiguration details.
You can also use a command-line argument interface as detailed morein the examples.I recommend you test this on small batches of data,
then stick it behind GNUparalleland feed the whole FASTQ file viazcatin on standard input.
This parallelizes with a small memory footprint, then
you write it out to disk (or stream into another tool).ThanksAgain, the tool is built upon on the excellent work ofregexBiopythonparallelDevelopment, helpingAny issues or advice are welcome as anissue on the gitlab repo.
Complaints are especially welcome.For development, see thedocumentation as rendered from docstrings.A set of tests is written up withpytestmodule, and can be run from inside
the cloned repo withmake test.
Seemake helpfor more options, such as building, installing, and uploading.There's also a bash script with some longer runs inprofiling_tests, these generate longer runs for profiling purposes
withcProfileandsnakeviz.
But is out of date. Todo is to re-configure and retest that for speed.