Package: hostile
HostileHostile accurately removes host sequences from short and long read (meta)genomes, consuming paired or unpairedfastq[.gz]input. Batteries are included – a human reference genome is downloaded when run for the first time. Hostile is precise by default, removing anorder of magnitude fewer microbial readsthan existing approaches while removing >99.5% of real human reads from 1000 Genomes Project samples. For the best possible retention of microbial reads, use an existing index masked against bacterial and/or viral genomes, or make your own using the built-in masking utility. Read headers can be replaced with integers (using--rename) for privacy and smaller FASTQs. Heavy lifting is done with fast existing tools (Minimap2/Bowtie2 and Samtools). Bowtie2 is the default aligner for short (paired) reads while Minimap2 is default aligner for long reads. In benchmarks, bacterial Illumina reads were decontaminated at 32Mbp/s (210k reads/sec) and bacterial ONT reads at 22Mbp/s, using 8 alignment threads. Further information and benchmarks can be found in thepaperandblog post. Please open an issue to report problemsorotherwisereachoutfor help, advice etc.Reference genomes (indexes)The default indexhuman-t2t-hlacomprisesT2T-CHM13v2.0andIPD-IMGT/HLAv3.51, and is downloaded automatically when running Hostile unless another index is specified. Slightly higher microbial sequence retention is possible using a masked index, of which several are available. The indexhuman-t2t-hla-argos985is masked against985 reference grade bacterial genomesincluding common human pathogens, whilehuman-t2t-hla-argos985-mycob140is further masked against mycobacterial genomes. To use a standard index, simply pass its name as the value of the--indexargument which takes care of downloading and cacheing the relevant index. Automatic download can be disabled using the--offlineflag, and--indexcan accept a path to a custom reference genome or Bowtie2 index.Object storageis provided by theModMedMicro research group.NameCompositionDateMasked positionshuman-t2t-hla(default)T2T-CHM13v2.0+IPD-IMGT/HLAv3.512023-070 (0%)human-t2t-hla-argos985human-t2t-hlamasked with 150mers for985FDA-ARGOSbacterialgenomes2023-07317,973 (0.010%)human-t2t-hla.rs-viral-202401_ml-phage-202401human-t2t-hlamasked with 150mers for 18,719 RefSeqviraland 26,928Millard Labphagegenomes2024-011,172,993 (0.037%)human-t2t-hla.argos-bacteria-985_rs-viral-202401_ml-phage-202401human-t2t-hlamasked with 150mers for985FDA-ARGOSbacterial, 18,719 RefSeqviral, and 26,928Millard Labphagegenomes2024-011,473,260 (0.046%)human-t2t-hla-argos985-mycob140human-t2t-hlamasked with 150mers for985FDA-ARGOSbacterial&140mycobacterialgenomes2023-07319,752 (0.010%)Performance ofhuman-t2t-hlaandhuman-t2t-hla-argos985-mycob140was evaluated in thepaperInstallInstallation with conda/mamba or Docker is recommended due to non-Python dependencies (Bowtie2, Minimap2, Samtools and Bedtools). Hostile is tested with Ubuntu Linux 22.04, MacOS 12, and under WSL for Windows.Conda/mambacondacreate-y-nhostile-cconda-forge-cbiocondahostile
condaactivatehostileDockerdockerrunquay.io/biocontainers/hostile:0.4.0--pyhdfd78af_0# Build your ownwgethttps://raw.githubusercontent.com/bede/hostile/main/Dockerfile
dockerbuild.--platformlinux/amd64Index installation (optional)Hostile automatically downloads and caches the default indexhuman-t2t-hlawhen run for the first time, meaning that there is no need to download an index in advance. Neverthless:To download and cache the default index (human-t2t-hla), runhostile fetchTo list available indexes, runhostile fetch --listTo download and cache another standard index, run e.g.hostile fetch --name human-t2t-hla-argos985To use a custom genome (made perhaps withhostile mask), runhostile clean --index path/to/genome.faCommand line usage$hostileclean-h
usage:hostileclean[-h]--fastq1FASTQ1[--fastq2FASTQ2][--aligner{bowtie2,minimap2,auto}][--indexINDEX][--invert][--rename][--reorder][--out-dirOUT_DIR][--threadsTHREADS][--aligner-argsALIGNER_ARGS][--force][--offline][--debug]Removereadsaligningtoatargetgenomefromfastq[.gz]inputfiles.

options:-h,--helpshowthishelpmessageandexit--fastq1FASTQ1pathtoforwardfastq[.gz]file--fastq2FASTQ2optionalpathtoreversefastq[.gz]file(default:None)--aligner{bowtie2,minimap2,auto}alignmentalgorithm.DefaultisBowtie2(pairedreads)&Minimap2(unpairedreads)(default:auto)--indexINDEXnameofstandardindexorpathtocustomindex(default:human-t2t-hla)--invertkeeponlyreadsaligningtothetargetgenome(andtheirmatesifapplicable)(default:False)--renamereplacereadnameswithincrementingintegers(default:False)--reorderensuredeterministicoutputorder(default:False)--out-dirOUT_DIRpathtooutputdirectory(default:/Users/bede/Research/Git/hostile)--threadsTHREADSnumberofalignmentthreads.Asensibledefaultischosenautomatically(default:5)--aligner-argsALIGNER_ARGSadditionalargumentsforalignment(default:)--forceoverwriteexistingoutputfiles(default:False)--offlinedisableautomaticindexdownload(default:False)--debugshowdebugmessages(default:False)Short reads, default index$hostileclean--fastq1human_1_1.fastq.gz--fastq2human_1_2.fastq.gz
INFO:Hostileversion1.0.0.UsingBowtie2(pairedreads)INFO:Foundcachedstandardindexhuman-t2t-hla(Bowtie2)INFO:Cleaning…
INFO:Cleaningcomplete[{"version":"1.0.0","aligner":"bowtie2","index":"human-t2t-hla","options":[],"fastq1_in_name":"human_1_1.fastq.gz","fastq1_in_path":"/Users/bede/human_1_1.fastq.gz","fastq1_out_name":"human_1_1.clean_1.fastq.gz","fastq1_out_path":"/Users/bede/human_1_1.clean_1.fastq.gz","reads_in":2,"reads_out":0,"reads_removed":2,"reads_removed_proportion":1.0,"fastq2_in_name":"human_1_2.fastq.gz","fastq2_in_path":"/Users/bede/human_1_2.fastq.gz","fastq2_out_name":"human_1_2.clean_2.fastq.gz","fastq2_out_path":"/Users/bede/human_1_2.clean_2.fastq.gz"}]Short reads, masked index, save log$hostileclean--fastq1human_1_1.fastq.gz--fastq2human_1_2.fastq.gz--indexhuman-t2t-hla-argos985>log.json
INFO:Hostileversion1.0.0.UsingBowtie2(pairedreads)INFO:Foundcachedstandardindexhuman-t2t-hla(Bowtie2)INFO:Cleaning…
INFO:CleaningcompleteLong reads$hostileclean--fastq1tests/data/tuberculosis_1_1.fastq.gz
INFO:Hostileversion1.0.0.UsingMinimap2'slongreadpreset
INFO:Foundcachedstandardindexhuman-t2t-hla(Minimap2)INFO:Cleaning…
INFO:Cleaningcomplete[{"version":"1.0.0","aligner":"minimap2","index":"human-t2t-hla","options":[],"fastq1_in_name":"tuberculosis_1_1.fastq.gz","fastq1_in_path":"/Users/bede/Research/Git/hostile/tests/data/tuberculosis_1_1.fastq.gz","fastq1_out_name":"tuberculosis_1_1.clean.fastq.gz","fastq1_out_path":"/Users/bede/Research/Git/hostile/tuberculosis_1_1.clean.fastq.gz","reads_in":1,"reads_out":1,"reads_removed":0,"reads_removed_proportion":0.0}]Python usagefrompathlibimportPathfromhostile.libimportclean_fastqs,clean_paired_fastqs# Long reads, defaultsclean_fastqs(fastqs=[Path("reads.fastq.gz")],)# Paired short reads, various options, capture loglog=clean_paired_fastqs(fastqs=[(Path("reads_1.fastq.gz"),Path("reads_2.fastq.gz"))],index="human-t2t-hla-argos985",out_dir=Path("decontaminated-reads"),rename=True,force=True,threads=4)print(log)Masking reference genomesThemasksubcommand makes it easy to create custom-masked reference genomes and achieve maximum retention of specific target organisms:hostilemaskhuman.fastalots-of-bacterial-genomes.fasta--threads8You may wish to use one of the existingreference genomesas a starting point. Masking uses Minimap2 to align 150mers of the supplied target genomes with the reference genome, and bedtools to mask all aligned regions with N. Both a masked genome (for Minimap2) and a masked Bowtie2 index is created.Known issuesUsing more than 10 alignment threads may reduce performance, even on systems with enough CPU cores. A sensible default is therefore chosen automatically at runtime. To maximise performance on your system, some experimentation may be necessary.Minimap2 has an overhead of 30-90s for human genome indexing prior to starting decontamination. Surprisingly, loading a prebuilt index is not significantly faster. I hope to mitigate this in a future release.CitationBede Constantinides, Martin Hunt, Derrick W Crook,  Hostile: accurate decontamination of microbial host sequences,Bioinformatics, 2023; btad728,https://doi.org/10.1093/bioinformatics/btad728@article{10.1093/bioinformatics/btad728,
    author ={Constantinides, Bede and Hunt, Martin and Crook, Derrick W},
    title ={Hostile: accurate decontamination of microbial host sequences},
    journal ={Bioinformatics},
    volume ={39},
    number ={12},
    pages ={btad728},
    year ={2023},
    month ={12},
    issn ={1367-4811},
    doi ={10.1093/bioinformatics/btad728},
    url ={https://doi.org/10.1093/bioinformatics/btad728},
    eprint ={https://academic.oup.com/bioinformatics/article-pdf/39/12/btad728/54850422/btad728.pdf},}Development installgitclonehttps://github.com/bede/hostile.gitcdhostile
condaenvcreate-y-fenvironment.yml
condaactivatehostile
pipinstall--editable'.[dev]'pytest