bigdatagenomics/adam — pull request #402 by fnothaft (2014-10-06 03:00:45+00:00)
Title: [ADAM-401] Adding read error correction.

Fixes #401, by adding a read error correction. This algorithm follows the following general approach:

1. Reads are cut into _q_-mers, which are quality score weighted _k_-mers
2. A mixture of gammas is fit to the data, where the mixture contains _m_ + 1 components (_m_ = ploidy). These components include an "error" class, _m_ - 1 heterozygous classes, and 1 homozygous class.
3. To each _q_-mer, we assign a probability that it _is_ a "true" _q_-mer, given the above mixture model.
4. To each read, we assign base probabilities by looking at the truth probabilities of all _k_-mers that cover the base.
5. From this, we collect MLEs for the transition probabilities of each base. Our covariates include the measured quality score of the base, the position of the base in the read, and the sequenced base.
6. These transition probabilities are used as priors for the fixing process. For each read, we compute the likelihoods of each base using the _q_-mer "truth" probabilities from step 3. We first compensate by the priors. After this, we try to fix bases in the read. We employ a coordinate ascent method, using the probabilities we calculate at each step as conjugate priors for each new fixing step.

On a small test, this appears to reduce the error rate by about an order of magnitude, as is expected. I'll be doing more tests this week, and will be putting together a writeup describing the method.