jamescasbon/PyVCF — Issue #25: Cythonize _parse_samples
Author: arq5x
Created: 2012-02-14 17:32:18+00:00
State: open

As expected, looping through the samples in _parse_samples is rather slow for files with many samples (e.g., [1]). It seems that the main bottlenecks are Python's split() and the fact that looping in Python is known to be dreadfully slow.

One potential solution would be to port the slowest looping bits to Cython. Another would be to work with BCF.

Thoughts? I think dealing with deep and wide files will be crucial for the future.

[1] ftp://ftp.1000genomes.ebi.ac.uk/vol1/ftp/technical/working/20120131_omni_genotypes_and_intensities/Omni25_genotypes_2141_samples.b37.vcf.gz