tdunning/t-digest — pull request #23 by jpountz (2014-04-28 12:30:46+00:00)
Title: Compress TDigest instances more aggressively.

TDigest instances are expected to track about 5*compression centroids. Yet they
get compressed only when the number of centroids grows larger than
100*compression.

I did some experiments with lower thresholds, and it seems to behave rather
well. In particular, with a threshold of 20*compression (computed empirically),
I get:
1. a slowdown of ~30% for sequential points (but memory usage is reduced by 5x)
2. a speedup that can be significant (especially for ArrayDigest, I have seen
speedups up to 50%) for data distributions whose amplitude is slowly
expanding.

All in all, I think lowering the threshold to 20*compression would be a better
trade-off?