Package: ngs-analysis
ngs-analysisConvenient analysis of sequencing reads that span multiple DNA or protein parts. For instance, given a library of protein variants linked to DNA barcodes, this tool can answer questions like:How accurate are the variant sequences, at the DNA or protein level?How frequently is the same barcode linked to two different variants?Which reads contain parts required for function (e.g., a kozak start sequence, or a fused protein tag)?This kind of analysis often involves parsing raw sequencing reads for DNA and/or protein sub-sequences (parts), then mapping the parts to a reference of anticipated part combinations. Here the workflow is:Define how to parse reads into parts using plain text expressions (no code)Parse your anticipated DNA sequences to generate a referenceParse a batch of sequencing samplesMap the parts found in each read to the referenceIt’s been tested with Illumina paired-end reads and Oxford Nanopore long reads. Under the hood it usesNGmergeto merge paired reads andMMseqs2for sequencing mapping. It is moderately performant: 1 million paired-end reads can be mapped to a reference of 100,000 variant-barcode pairs in ~1 minute.WorkflowA cartoon example with two reference sequences, each consisting of a variant linked to a barcode:Here's the analysis workflow and outputs:Note that in the last two columns, the parsed variant is mapped to a reference variant defined by the barcode present in the same read, rather than all possible reference variants. Check out theexample notebook for paired end readsfor details.TL;DRRunngs-analysis --helpto see available commands.Make an empty directory, addconfig.yamlandsamples.csvbased on the example.Addreference_dna.csvwith anticipated DNA sequences (including adapters).Runngs-analysis setup. Add--cleanto start the analysis from scratch.Check thatdesigns.csvis accurate; if not, fixconfig.yaml.If you have paired-end data, put it in0_paired_reads/and runngs-analysis merge_read_pairs <sample>.If you have single-end data (e.g., nanopore), put it in1_reads/.Runngs-analysis parse_reads <sample>. Check that2_parsed/<sample>.parsed.pqlooks alright (with pandas, usepd.read_parquet)Runngs-analysis map_parsed_reads <sample>. Results are in3_mapped/<sample>.mapped.csvSimulation modeDebugging complex read structures and experimental layouts can be tricky. For example, yourconfig.yamlmight parse reference sequences incorrectly, or samples might map to the reference in an unexpected way (e.g., if the same barcode is attached to different variants).Before running an analysis (or designing an experiment), you can simulate the results by definingsample_plan.csvand runningsimulate_single_readsorsimulate_paired_reads, which have options to add simple random mutations and variable coverage per subpool.Here'ssample_plan.csvfrom thepaired read example. Note that "source" refers to the optional "source" column inreference_DNA.csv.samplesourcecoveragesample_Apool150sample_Bpool250sample_Cpool150sample_Cpool220sample_Dpool350To analyze the simulated data, just add the--simulateflag tomerge_read_pairs,parse_reads,map_parsed_reads, andplot. Results will be saved to{step}/simulate/{sample}rather than{step}/{sample}.Installpipinstallngs-analysisMake sure that themmseqsandNGmergeexecutables are available (NGmerge is only needed for paired reads).On Linux and Intel-based MacOS, you can useconda install -c bioconda -c conda-forge mmseqs2 ngmerge. On Apple Siliconmmseqscan be installed via Homebrew withbrew install mmseqs2, and NGmerge can be installedfrom source, or viabrew install brewsci/bio/ngmerge.Tested on Linux and MacOS (Apple Silicon).