A faster implementation of the Plink Bed Reader that's optimized for SNP-major bed files. Reduced runtime from 102 minutes to 12 minutes. Partially tested.

Recover gracefully from completely correlated haplotype data: run Penalized IRLS to prevent divergence (usually one step). Tested.
Evaluating the logistic regression given a large matrix of predictors was a killer roadblock (>99% runtime). NumPy's applyOverDimension didn't cut it. Instead: write the appropriate function as a c extension and call into it.

This is perhaps the most painful thing ever with python, but totally doable (better than a .mex file with matlab -- eh Guillermo?)
Alter the convergence criteria not to focus on beta (which we don't care about here) but rather on the convergence of the predictors. This leads to far quicker convergence in the case where some responses can be completely explained (so some of the true beta values are infinite).
Next big holdup: numpy.diag(N*p*(1-p)) is way too slow. Place this into a c routine as well.

Also, after much debugging, the memory leaks and segfaults (well...those that I know about so far) are coming from numpy.linalg.lstsq. Give up on the guessing, and set beta to 0 initially.
Some final touches (pointers --> variables).

With this we've gone from:

ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.034    0.034  483.779  483.779 <string>:1(<module>)

to:

ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.041    0.041   31.006   31.006 <string>:1(<module>)

or a reduction of the runtime by 94%.

notbad.gif
Merge branch 'master' of gsa2:/humgen/gsa-scr1/chartl/dev/unstable
Merge branch 'master' of gsa2:/humgen/gsa-scr1/chartl/dev/unstable
Merge branch 'master' of gsa2:/humgen/gsa-scr1/chartl/dev/unstable
Merge branch 'master' of gsa2:/humgen/gsa-scr1/chartl/dev/unstable
Merge branch 'master' of gsa2:/humgen/gsa-scr1/chartl/dev/unstable
Merge branch 'master' of gsa2:/humgen/gsa-scr1/chartl/dev/unstable
Merge branch 'master' of gsa2:/humgen/gsa-scr1/chartl/dev/unstable
Merge branch 'master' of gsa2:/humgen/gsa-scr1/chartl/dev/unstable
Merge branch 'master' of gsa2:/humgen/gsa-scr1/chartl/dev/unstable
Merge branch 'master' of gsa2:/humgen/gsa-scr1/chartl/dev/unstable
Merge branch 'master' of gsa2:/humgen/gsa-scr1/chartl/dev/unstable
Merge branch 'master' of gsa2:/humgen/gsa-scr1/chartl/dev/unstable
Add removeCall, which removes all calls matching the provided MongoVariantContext. _id and Date fields are excluded. Generally intended te remove only a single call
some changes.

some changes.
removed comments and uneeded imports