Package: bcf-extras
bcf_extrasExtra variant file helper utilities built on top ofbcftoolsandhtslib.Licensebcf_extrasis licensed under GPLv3. Seethe licensefor more
information.DependenciesThe package requires thatbcftoolsandhtslibare installed on your
operating system.For thestrextra,TRtoolsis required as a pip dependency. This should be
handled automatically upon installation, but I have encountered installation
issues before (especially when copies ofhtslibinterfere with one another.)
Feel free to file an issue to ask about this.InstallationOne can use pip to install either the basebcf-extrasorbcf-extras[str],
which has additional dependencies and adds some more utilities related to STR
calling.pipinstallbcf-extras# orpipinstallbcf-extras[str]What's Included (Base)copy-compress-indexCreates.vcf.gzfiles with a corresponding tabix indices from one or more
VCFs, sorting the VCFs if necessary.For example, the following would generatesample-1.vcf.gzandsample-1.vcf.gz.tbi:bcf-extrascopy-compress-indexsample-1.vcfadd-header-linesAdds header lines from a text file to a particular position in the VCF header.
Useful for e.g. inserting missing##contiglines to a bunch of VCFs at once
(taking advantage of this + something like GNU parallel.)For the##contiglines example, inserting the contents oftests/vcfs/new_lines.txt, we could run the
following command ontests/vcfs/ahl.vcf, replacing the
file with a new copy:bcf-extrasadd-header-linestests/vcfs/ahl.vcftests/vcfs/new_lines.txtThere is also a flag,--tmp-dir, for specifying a temporary folder location
into which header artifacts will be placed. This is especially useful when
running jobs on clusters, which may have specific locations for temporary I/O.Using GNU parallel, we can do multiple VCFs at once, e.g.:parallel'bcf-extras add-header-lines {} tests/vcfs/new_lines.txt --keep-old':::/path/to/my/vcfs/*.vcfThe--keep-oldflag keeps the original VCFs as a copy.arg-joinSome bioinformatics utilities take in comma-separated file lists rather than
the more standard whitespace-separated lists that something like a glob
(*.vcf.gz) generates.This command can be run by itself, e.g.:bcf-extrasarg-join--sep";"*.vcf# Outputs e.g. sample1.vcf;sample2.vcfIt can be used embedded in another command, e.g. withmergeSTR,a tool for merging STR caller VCFswhich takes as input acomma-separatedlist of files:mergeSTR--vcfs$(bcf-extrasarg-join*.vcf)--outmy_mergeThe default separator (specified via--sep) is,.filter-gff3This command can filter a GFF3 (or similarly formatted) file and filter it
by various columns using regular expressions.It prints the filtered lines tostdout, which can then be redirected to a
file or piped to another process.Currently, you can filter by theseqid,source,type,strand, andphrasecolumns using Python-formatted regular expressions, e.g. the
following, which filterstypeto be eithergeneorexonand stores that
in a new file:bcf-extrasfilter-gff3--type'^(gene|exon)$'example.gff3>example-genes-exons.gff3For help, run the sub-command with no arguments:bcf-extrasfilter-gff3reformat-vcf-contigsTODOWhat's Included (STR)parallel-mergeSTRmergeSTRis a tool by the Gymrek lab
used to merge STR call VCFs. It proceeds linearly over a list of files, which
cannot easily take advantage of multiple cores. This utility merges VCFs in a
tree fashion to produce a final merged result, and is handy when merging 100s
of STR call VCFs at once.bcf-extrasparallel-mergeSTR*.vcf.gz--outmy_merge--ntasks10In a dataset of 148 single-sample gangSTR call VCFs, merging withparallel-mergeSTRon 10 cores resulted in an 60% speedup versus
running on a single core (~2 hours versus ~5 hours.)Speedup is not linear with number of cores used, so it only makes sense to use
this if turnaround time is important and resources are available.To not over-allocate resources on a cluster, the process can be split further
into first a parallelized task, and then a task which only uses one core:# Runs on multiple coresbcf-extrasparallel-mergeSTR*.vcf.gz--ntasks10--outmy_merge--step1-only# Intermediate files generated by the first step will feed into the second step.# Bottlenecked single process step; the ntasks argument is still needed to# calculate the names of the intermediate output files (but sub-processes are# not spawned) - thus, the provided value must match the value above.bcf-extrasparallel-mergeSTR--ntasks10--outmy_merge--step2-only