lh3/readfq — pull request #4 by vals (2013-04-10 17:04:29+00:00)
Title: Slight speedup of readfq.py

This small change gives a speed increase by not searching through an
entire string for splitting when we're only interested in the first
element after the split.

To test the change I used a small fastq file I had lying around. But I inputted it several times
by repeating it as an argument to `cat`.

$ FQ = /path/to/my/example.fastq

and then

$ time cat $FQ $FQ $FQ $FQ $FQ | python readfq.py
7070210	985666670	985666670

real	0m30.388s
user	0m30.010s
sys	0m1.186s

I gathered the results for four runs of this before and after the change. Here are the times.

This is running Python 2.7.2

.split()[0]	.partition(" ")[0]
real	user	sys	real	user	sys
0m30.388s	0m30.010s	0m1.186s	0m28.579s	0m27.868s	0m1.063s
0m29.712s	0m29.484s	0m1.050s	0m27.819s	0m27.530s	0m1.156s
0m29.795s	0m29.497s	0m1.149s	0m27.754s	0m27.413s	0m1.057s
0m30.718s	0m30.533s	0m1.050s	0m27.653s	0m27.290s	0m1.072s

The speedup is quite small, but scales with the number of reads.

I also compared using the two different string splitting methods when running PyPy.
When comparing PyPy execution times it is a bit trickier as the JIT will make each
successive run of a script quicker. But I believe these numbers reflect realistic
situations where a script using `readfq` would be used.

This is all run as

$ time cat $FQ $FQ $FQ $FQ $FQ | pypy readfq.py

for PyPy 1.9.0

.split()[0]	.partition(" ")[0]
real	user	sys	real	user	sys
0m18.568s	0m17.911s	0m1.831s	0m17.305s	0m16.666s	0m1.817s
0m18.509s	0m17.841s	0m1.853s	0m17.292s	0m16.603s	0m1.872s
0m18.514s	0m17.738s	0m1.959s	0m17.354s	0m16.737s	0m1.806s
0m18.530s	0m17.964s	0m1.754s	0m17.293s	0m16.594s	0m1.882s

Again, the speed increase is not tremendous, but this is only ~7 million reads, and it
scales over the number of reads.