Rapleaf/hank — Issue #225: Minimize number of random record file reads during Curly compaction
Author: thomas-kielbus
Created: 2011-12-08 01:20:52+00:00
State: open

While we perform the key file merge sort, we do random reads of records to be written from the record files being merged. It seems like there is a lot of potential to optimize this process. We could merge sort a fixed number of keys (e.g. 1024) and then only read once from each record file and get all the necessary records in memory before writing them to the compacted partition.