clearspring/stream-lib — pull request #47 by whiter4bbit (2013-08-01 16:40:12+00:00)
Title: Attempt to decrease qdigest memory footprint

Hi,
Full discussion here https://groups.google.com/forum/#!topic/stream-lib-user/DuB-n4rU15Y.

I created benchmark for inserts: https://github.com/whiter4bbit/qdigest-bench. This benchmark does sequential inserts of 50000000 elements (`random.nextInt(120000)`).

whiter4bbit@katrin stream-lib-fork % java -version
java version "1.7.0_21"
Java(TM) SE Runtime Environment (build 1.7.0_21-b11)
Java HotSpot(TM) 64-Bit Server VM (build 23.21-b01, mixed mode)

Java options: `-server -Xmx1280m -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -Xloggc:qdigest.log`

I measured young collections count and young/old gen heap size at the end of each pass. Implementation, which uses Long2LongOpenHashMap don't have young gen collections at all (as well as old gen).

Results:

2.5.0-SNAPSHOT:

whiter4bbit@katrin qdigest-benchmark % bin/bench.sh 10 | ruby bin/summarize.rb

Insert
Mean: 45238
Stddev: 7632.287599402947
Median: 42502.0

Young gen GC count
Mean: 0
Stddev: 0.0
Median: 0.0

Young gen heap used (at the end)
Mean: 1885
Stddev: 0.0
Median: 1885.0

Old gen heap used (at the end)
Mean: 0
Stddev: 0.0
Median: 0.0

2.4.0:

whiter4bbit@katrin qdigest-benchmark % bin/bench.sh 10 | ruby bin/summarize.rb
Insert
Mean: 49892
Stddev: 2827.6702424434147
Median: 50452.0

Young gen GC count
Mean: 999
Stddev: 61.66846844214635
Median: 1021.0

Young gen heap used (at the end)
Mean: 39305
Stddev: 24565.86045714662
Median: 34994.5

Old gen heap used (at the end)
Mean: 486
Stddev: 20.784609690826528
Median: 488.0