ged-lab/khmer — pull request #621 by qingpeng (2014-09-28 01:52:00+00:00)
Title: add estimate_optimal_hash.py

initial effort to fix #390

Now I wrote a separate script to calculate optimal parameters of hash tables. There are two functions inside - estimate_optimal_with_N_and_M(N,M) and estimate_optimal_with_N_and_f(N,f).

To calculate minimum required memory to get desired false positiver rate, I noticed the formula described in khmer-counting paper was not accurate, not wrong though. Because optimal number of hash tables - Z must be integer. Practically The corresponding size of hash tables can not be calculated by log_0.6185(f)*N/Z, as described in khmer-counting paper. That is just the theoretical calculation on the premise that Z(number of hash tables) can be any number(including non-integer).

As to integrating the functions into khmer.khmer_args, my concern is that the number of distinct k-mers is difficult for end-user to estimate beforehand. One potential improvement is that we can supply a table of number of distinct k-mers in some typical sample datasets to give the end-user an idea. Like the number of distinct k-mers in e.coli genome, in human genome, in human gut metagenome, etc.