Package: poreplex
PoreplexSignal-level preprocessor for Oxford Nanopore direct RNA sequencing (DRS) data.
Poreplex does many preprocessing steps required before the downstream analyses
for RNA Biology and yields the processed data in the ready-to-use forms.FeaturesDemultiplexing barcodeddirect RNAsequencing librariesTrimming 3′ adapter sequencesFiltering pseudo-fusion readsBasecalling with ONTalbacore(even faster thanalbacoreitself)Live basecalling and processingduringthe sequencingReal-time read alignments withminimap2Full-screen dashboard view for real-time reportsVarious ready-to-use output formatsInstallationPoreplexrequires Python 3.5+ andpipto install.
Thispipcommand installsporeplexwith its essential dependencies. You may use the
following command.pipinstallporeplexTo install it together with all optional dependencies (exceptalbacore), use this
command:pipinstall'poreplex[full]'Additional (Optional) DependencyAs its inputs,poreplexrequires the FAST5 files that were basecalled usingONTalbacorein advance.
Alternatively,poreplexcan also internally callalbacoreduring the
processing without a prior basecalling if thealbacorepackage is
available from the environment.Quick StartProduce FASTQ files without 3′ adapter sequences from a bunch of FAST5 files.poreplex-ipath/to/fast5-opath/to/output--trim-adapterFour direct RNA sequencing libraries can be barcoded, pooled and sequenced
together.Porplexcan demultiplex the librariess into separate FASTQ files.poreplex-ipath/to/fast5-opath/to/output--trim-adapter--barcodingIn addition,poreplexcan create directories containing hard-links to
the original FAST5 files, organized separately by the barcodes.poreplex-ipath/to/fast5-opath/to/output--trim-adapter--barcoding--fast5In case the FAST5 files are not basecalled yet, just a switch letsporeplexcallalbacoreinternally. Multicore machines help.poreplex-ipath/to/fast5-opath/to/output--trim-adapter--barcoding--fast5--basecall--parallel40With the--liveswitch, All tasks can be processed as soon as reads
are produced from MinKNOW.poreplex-ipath/to/fast5-opath/to/output--trim-adapter--barcoding--basecall--parallel40--liveOne may want to outputalignedreads directly to BAM files instead of
FASTQ outputs. Poreplex streams the processed reads tominimap2and update
the BAM outputs real-time. A pre-built index (not a FASTA) generated usingminimap2must be provided for this.poreplex-ipath/to/fast5-opath/to/output--trim-adapter--barcoding--basecall\--parallel40--live--alignGRCz11-transcriptome.mmidxMore vibrant feedback is provided if you turn on the dashboard switch.poreplex-ipath/to/fast5-opath/to/output--trim-adapter--barcoding--basecall\--parallel40--live--alignGRCz11-transcriptome.mmidx--dashboardPoreplexdetects pseudo-fusion reads which may originate from
insufficiently segmented signals if--filter-chimerais given. This
improves the overall accuracy of demultiplexing.poreplex-ipath/to/fast5-opath/to/output--filter-chimeraBarcoding direct RNA sequencing librariesThe official kits and protocols do not support barcoding in the direct RNA
sequencing yet.Poreplexenables pooling multiple libraries into a single
DRS run.ONT direct RNA sequencing libraries are prepared by subsequently attaching
two different 3' adapters,RTA and RMX,
respectively. Both are double-stranded DNAs with Y-burged ends on the
3'-sides. Barcoded libraries forporeplexcan be built with modified versions of
RTA adapters. Unlike in the DNA sequencing libraries,poreplexdemultiplexes
in signal-level to ensure the highest accuracy. Theporeplexpackage
comes with pre-trained demultiplexer models for four different DNA barcodes.
Order these sequences as many as you need in the experiment and replace the
original RTA adapters.BC1 Oligo A:5'-/5Phos/CCTCCCCTAAAAACGAGCCGCATTTGCGTAGTAGGTTC-3'BC1 Oligo B:5'-GAGGCGAGCGGTCAATTTTCGCAAATGCGGCTCGTTTTTAGGGGAGGTTTTTTTTTT-3'BC2 Oligo A:5'-/5Phos/CCTCGTCGGTTCTAGGCATCGCGTATGCTAGTAGGTTC-3'BC2 Oligo B:5'-GAGGCGAGCGGTCAATTTTGCATACGCGATGCCTAGAACCGACGAGGTTTTTTTTTT-3'BC3 Oligo A:5'-/5Phos/CCTCCCACTTTCACACGCACTAACCAGGTAGTAGGTTC-3'BC3 Oligo B:5'-GAGGCGAGCGGTCAATTTTCCTGGTTAGTGCGTGTGAAAGTGGGAGGTTTTTTTTTT-3'BC4 Oligo A:5'-/5Phos/CCTCCTTCAGAAGAGGGTCGCTTCTACCTAGTAGGTTC-3'BC4 Oligo B:5'-GAGGCGAGCGGTCAATTTTGGTAGAAGCGACCCTCTTCTGAAGGAGGTTTTTTTTTT-3'Basecalling with the ONT AlbacoreMost studies requiring signal-level analysis need re-basecalling with
the ONTalbacoreto get the event tables in the FAST5 files.Poreplexcan internally call the basecaller core routines ofalbacoreto yield
the sequences and tables for the downstream analyses. In fact, runningalbacoreviaporeplexis remarkably faster than runningalbacoreitself in a multi-core machine thanks to more efficient scheduling of
the computational loads.Live basecalling and processingOne can start theporeplexpipeline at any time even before the
sequencing begins. With the--liveswitch turned on, it monitors
every update in input directories and picks the newly created
files up for the whole process of the analysis. In the live mode,
the program keeps running unless a user presses Ctrl-C (in the
standard progress view) or Q (in the full-screen dashboard view). Theinotifymodule is required to allowporeplexto run in the live mode.In case the points of sequencing and analysis are different,
a real-time directory synchronization software likeDirSync
Promay help.Poreplexdetects new files
introduced by moving or closing a file after writing. Files that are
made visible by creating hard or symbolic links or changing permissions
may remain undetected.Real-time sequence alignmentsPoreplexaligns the reads to a reference transcriptome usingminimap2and writes the results to BAM files when an index file for
the reference is provided. Some options that affect the performance of
the alignments can be specified when generating theminimap2index.wget'ftp://ftp.ebi.ac.uk/pub/databases/gencode/Gencode_human/release_28/gencode.v28.transcripts.fa.gz'minimap2-H-k13-w10-dgencode.v28.transcripts.mmidxgencode.v28.transcripts.fa.gz
poreplex-i/path/to/input-o/path/output--basecall--aligngencode.v28.transcripts.mmidxBy default, switching on the alignments suppresses the FASTQ
outputs. Those can be recovered by adding--fastqto the command line.Real-time reportsThe results from the real-time alignments with the overall progression of
the pipeline can be visualized as afull-screen dashboard view in a text
terminal.Poreplexshows the real-time report when the
command line includes the--dashboardswitch along with--alignfor
the index of the reference transcriptome. The names of mapped sequences
are shown as the sequence name in the referenceminimap2index. To
see them as more familiar names, supply a file containing IDs and names
with the--contig-aliasesswitch. It must be a tab-separated
text file with two columns. The first should contain IDs (in the reference index)
and the second should contain names (to show in the screen). The read counts
window in the middle of the screen represents the summary of reads categorized by
error status or detected barcodes for multiplexed libraries. A user can
choose a group to show in the window with up and down arrow keys. To stop the
process and close the dashboard, press Q key at any time.Pseudo-fusion filterIn the Oxford Nanopore strand sequencing, a read is a snippet from a
very long contiguous signal from a channel. In most cases, there is a gap
between two different molecules. The gap should be long enough for theMinKNOWto cut signals at the end of sequences. However, the gap between strands
is sometimes not enough, so that a small fraction of reads carry two or more molecules.
This phenomenon can be particularly problematic in the pooled libraries with barcodes
and fusion gene studies. In a few runs in our testing, up to 1% of reads could
be derived from insufficiently segmented signals. The following plot shows a signal
sequence continued without any gap between the ends of two differently
adapted RNAs which were prepared independently until the second ligation
step, before which RNAs were pooled (RT before the ligation includes the heat-
inactivation of enzymes).Poreplexdetects potential artifacts by detecting multiple appearances of the
signature of the DNA adapter in a single read when the--filter-chimeraswitch
is turned on. Note that the default parameters for the filtering can be
too sensitive for some experiments.Output formatsFASTQSequences and quality scores are written tobgzip-compressed
FASTQ files in thefastqsubdirectory. Each FASTQ file contains the
entire sequences of a group classified by processing status and
detected barcode.File nameDescriptionfastq/pass.fastq.gzAll sequences that were basecalled and passed the basic quality filters inporeplex. With--barcoding, the passed sequences with no identifiable barcode are written to this file.fastq/BC#.fastq.gzSequences with identifiable barcode signals.fastq/fail.fastq.gzToo short sequences that could not be calibrated for the signal processing.fastq/artifact.fastq.gzSequences that were classified as potential artifacts.FASTQ outputs are suppressed when BAM outputs are activated with
the--alignoption. Please add--fastqto restore the FASTQ outputs.FAST5To reduce the disk I/O,poreplexutilizes the links instead of copying
the original FAST5 to append basecalled results to the file. With the--fast5option,poreplexcreates hard links of the original FAST5
files reorganized in subdirectories representing each processing status
or barcode. Symbolic links are created in case the hard links are not
possible or--symlink-fast5is specified.The basecalled events, which are stored inAnalyses/Basecall_1D_00*of the standard FAST5 files, are written to theeventssubdirectory
instead upon request by--dump-basecalled-events. The basecall
event tables for all reads are accessible through a single HDF5 file,events/inventory.h5, by the read id. These tables include an
additionalscaled_meancolumn, which contains mean current levels
scaled to match the ONT's referencepore model.BAMThe alignments to the reference transcriptome go into BAM files
inside thebamsubdirectory. The reference sequences must be
indexed usingminimap2before providing it with the--alignoption (see above). The BAM
files are not sorted and not filtered thoroughly. FASTQ or FASTA
sequence files can be generated from the BAM files without loss
usingbedtools.
Please use these sequence alignments in the BAM files for quality
checks and sketchy analyses only.Nanopolish databaseNanopolishprovides very convenient
tools that help signal-level analyses.Poreplexprovides a set of index
files that are required to run thenanopolishcommands. Add--nanopolishto aporeplexcommand line, then just skipnanopolish extractornanopolish indexcommands in its tutorial, and proceed directly to
the main steps.Command line optionsusage:poreplex-iDIR-oDIR[-cNAME][--trim-adapter][--minimum-lengthLEN][--filter-chimera][--barcoding][--polya][--basecall][--alignINDEXFILE][--live][--live-delaySECONDS][--fastq][--fast5][--symlink-fast5][--nanopolish][--dump-adapter-signals][--dump-basecalled-events][--dashboard][--contig-aliasesFILE][-q][-y][-pCOUNT][--tmpdirDIR][--batch-sizeSIZE][--version][-h]Short optionLong optionDescriptionData Settings-i DIR--input DIRpath to the directory with the input FAST5 files (required)-o DIR--output DIRoutput directory path (Required)-c NAME--config NAMEpath to signal processing configurationBasic Processing Options--trim-adaptertrim 3′ adapter sequences from FASTQ outputs--filter-chimeraremove unsplit reads fused of two or more RNAs in outputOptional Analyses--barcodingsort barcoded reads into separate outputs--polyaoutput poly(A) tail length measurements--basecallcall the ONT albacore for basecalling on-the-fly--align INDEXFILEalign basecalled reads using minimap2 and create BAM filesLive Mode--livemonitor new files in the input directory--live-delay SECONDStime to delay the start of analysis in live mode (default: 60)Output Options--fastqwrite to FASTQ files even when BAM files are produced--fast5link or copy FAST5 files to separate output directories--symlink-fast5create symbolic links to FAST5 files in output directories even when hard linking is possible--nanopolishcreate a nanopolish readdb to enable access from nanopolish--dump-adapter-signalsdump adapter signal dumps for training--dump-basecalled-eventsdump basecalled events to the outputUser Interface--dashboardshow the full screen dashboard--contig-aliases FILEpath to a tab-separated text file for aliases to show as a contig names in the dashboard (see README)-q--quietsuppress non-error messages-y--yessuppress all questionsPipeline Options-p COUNT--parallel COUNTnumber of worker processes (default: 1)--tmpdir DIRtemporary directory for intermediate data--batch-size SIZEnumber of files in a single batch (default: 128)--versionshow program's version number and exit-h--helpshow this help message and exitCiting PoreplexA pre-print is going to be uploaded soon.