Package: loreme
LoReMe pipelineTODO:Document setting pore type/chemistryTidy up rough edges in documentation (e.g. cli reference)Review handling of ONT bedMethyl formatFind out why Modkit runs with only one threadRust-ify postprocessing functionsunit tests for postprocessing functionsLoReMe (Long Read Methylaton) is a Python package facilitating analysis of
DNA methylation signals fromPacific BiosciencesorOxford Nanoporelong read sequencing data.It consists of an API and CLI for three distinct applications:Pacific Biosciences data processing. PB reads in SAM/BAM format are aligned
to a reference genome with the special-purpose alignerpbmm2,
a modified version ofminimap2.
Methylation calls are then piled up from the aligned reads withpb-CpG-tools.Oxford nanopore basecalling. ONT reads are optionally converted from FAST5
toPOD5format, then
basecalled and aligned to a reference withdorado(dorado alignment also uses minimap2 under the hood), and finally piled up withmodkit.Postprocessing and QC of methylation calls. Several functions are available
to generate diagnostic statistics and plots.See also thefull documentation.Other tools of interest:methylartist,modbamtools(modbamtools docs),methplotlibInstallationIn a Conda environmentThe recommended way to installloremeis with a dedicatedcondaenvironment:First create an environment including all dependencies:condacreate-nloreme-cconda-forge-cbiocondasamtoolspbmm2\urllib3pybedtoolsgff2bedseabornpyfaidxpsutilgputiltabulate\cythonh5pyiso8601more-itertoolstqdm
condaactivateloremeThen install withpip:pipinstallloremeYou may also wish to installnvtopto monitor GPU usage:condainstall-cconda-forgenvtopWith pippipinstallloremeCheck installationCheck that the correct version was installed withloreme --versionUninstallTo uninstall loreme:loremeclean
pipuninstallloremeOxford Nanopore readsDownload example datasetUseloreme download-ont-exampleto download aK. azureus(zebra perch) genome assembly, gene annotations, and a POD5 file containing ONT reads.NoteThis example analysis involves downloading and processing a 26 GB POD5 file of nanopore reads, which will be time consuming and resource-intensive. For that reason, it is recommended to reproduce it on a remote server (such as Seabiscuit for members of the Michael Lab at Salk).mkdir loreme_example_Kazu/
loreme download-ont-example -d loreme_example_Kazu/Consider going for a coffee break with your colleagues, as this will take a few minutes. Once the download is complete, the example datasets will be present in the desintation directory.cd loreme_example_Kazu/
lsKazu_090722.primary_high_confidence.gff3.gz
Kazu_090722.softmasked.fasta.gz
Kazu_090722_95etoh_22C_6w_SRE1_deep.pod5Download doradoCalling methylation from ONT long reads requires the basecallerdorado. Download it by runningloreme download-dorado <platform>This will download dorado and several basecalling models. The platform should be one of:linux-x64,linux-arm64,osx-arm64,win64, whichever matches your system. Runningloreme download-dorado --helpwill show a hint as to the correct choice.NoteFor members ofMichael Labat Salk running on Seabiscuit, useloreme download-dorado linux-x64.Modified basecallingNoteIf you have FAST5 data, convert it to POD5 usingloreme dorado-convert(seeloreme dorado-convert --help)The POD5 file will need to be contained in its own directory on a scratch disk
to be basecalled, so create a directory and move it there:NoteSince basecalling ONT data is disk-read intensive, it will be slow on a spinning disk. The environment variable$SCRATCHshould be set to a location on a fast SSD or NVMe disk for the following steps. For Michael lab members running on Seabiscuit, use this command:SCRATCH=/scratch/<username>.mkdir $SCRATCH/Kazu_090722_pod5/
mv Kazu_090722_95etoh_22C_6w_SRE1_deep.pod5 $SCRATCH/Kazu_090722_pod5/First check that a GPU is available to use:loremecheck-gpuYou can carry out modified basecalling (i.e. DNA methylation) with default parameters by running:loreme dorado-basecall $SCRATCH/Kazu_090722_pod5/ $SCRATCH/Kazu_090722.bamThe input argument should be a directory containing one or more POD5 files. The output argument is a BAM file containing MM/ML tags. For other parameter options, seeloreme dorado-basecall --helpNoteBy default, LoReMe uses the "fast" basecalling model, which is less accurate. In practice you will probably want to use a higher accuracy model by setting the--accuracyoption, e.g.loreme dorado-basecall --accuracy sup $SCRATCH/Kazu_090722_pod5/ $SCRATCH/Kazu_090722.bamNoteWhile basecalling, tools likehtopormutilmay show your memory usage increasing to hundreds of gigabytes. This is a bug in dorado and/or how Linux communicates with the GPU, and does not reflect actual memory usage.To run dorado with only regular basecalling, use the--no-modoption:loreme dorado-basecall --no-mod <pod5s/> <output.bam>AlignmentThe BAM file produced by dorado can be aligned to a reference index (FASTA or MMI file) withloreme dorado-align:loreme dorado-align Kazu_090722.softmasked.fasta.gz $SCRATCH/Kazu_090722.bam Kazu_090722_aligned.bamDownload modkitPiling up methylation calls from BAM data requiresmodkit. Download it by running:loreme download-modkitCheck BAM file for MM/ML tagsBefore processing with modkit, check that the BAM file contains MM/ML tags.loremecheck-tagsKazu_090722_aligned.bamPileupThe pileup step generates a bedMethyl file from an aligned BAM file.Before running modkit, you will need to decompress the FASTA file.gunzip Kazu_090722.softmasked.fasta.gzThen you can generate a pileuploreme modkit-pileup Kazu_090722.softmasked.fasta Kazu_090722_aligned.bam Kazu_090722.bedNoteSeeloreme modkit-pileup --helpfor additional options. On a HPC system you may want to use additional threads with the-tflag.PostprocessingSee also thePacific Biosciences readssection for examples of postprocessing analysis that can be applied to bedMethyl files.Calculate mean methylation levelloreme mean --total Kazu_090722.bedGroup        Methylation level (%)
0    79.64114117704054NoteThis and subsequent steps represent postprocessing that can be applied to either Pacific Biosciences or Oxford Nanopore datasets.Methylation level of promotersTo calculate methylation levels of promoter regions:loremepromoterKazu_090722.primary_high_confidence.gff3.gz\Kazu_090722.bed--histKazu_090722_promoter.svg>Kazu_090722_promoter.bed
headKazu_090722_promoter.bedctg1	159758	161758	Kazu_090722.ctg1.g000010	86.36	-
ctg1	256072	258072	Kazu_090722.ctg1.g000030	64.29	+
ctg1	321679	323679	Kazu_090722.ctg1.g000050	76.79	-
ctg1	397819	399819	Kazu_090722.ctg1.g000060	45.23	+
ctg1	437295	439295	Kazu_090722.ctg1.g000070	100.00	+
ctg1	444908	446908	Kazu_090722.ctg1.g000080	59.09	-
ctg1	499196	501196	Kazu_090722.ctg1.g000100	97.64	-
ctg1	547954	549954	Kazu_090722.ctg1.g000110	100.00	+
ctg1	577225	579225	Kazu_090722.ctg1.g000120	77.38	-
ctg1	662637	664637	Kazu_090722.ctg1.g000130	100.00	-Histogram of methylation levels of promoter regions:Methylation level of gene bodiesTo calculate methylation levels of gene bodies:loremegene-bodyKazu_090722.primary_high_confidence.gff3.gz\Kazu_090722.bed--histKazu_090722_gene_body.svg>Kazu_090722_gene_body.bed
headKazu_090722_gene_body.bedctg1	159267	159758	Kazu_090722.ctg1.g000010	95.00	-
ctg1	237075	241223	Kazu_090722.ctg1.g000020	92.05	-
ctg1	258072	259127	Kazu_090722.ctg1.g000030	100.00	+
ctg1	320906	321679	Kazu_090722.ctg1.g000050	66.67	-
ctg1	399819	411211	Kazu_090722.ctg1.g000060	83.36	+
ctg1	439295	440181	Kazu_090722.ctg1.g000070	77.78	+
ctg1	444411	444908	Kazu_090722.ctg1.g000080	100.00	-
ctg1	456252	477732	Kazu_090722.ctg1.g000090	89.00	+
ctg1	498921	499196	Kazu_090722.ctg1.g000100	100.00	-
ctg1	549954	558103	Kazu_090722.ctg1.g000110	98.23	+Histogram of methylation levels of gene bodies:Gene methylation profileTo plot a profile of methylation across gene bodies:loremeplot-genesKazu_090722.primary_high_confidence.gff3.gz\Kazu_090722.bedKazu_090722_genes.svgMethylation profile across gene bodies: