apache/spark — pull request #2134 by liyezhang556520 (2014-08-26 10:36:57+00:00)
Title: [SPARK-3000][CORE] drop old blocks to disk in parallel when memory is no...

...t large enough for caching new blocks

Currently, old blocks dropping for new blocks' caching are processed by one thread at the same time. Which can not fully utilize the disk throughput. If the to be dropped block size is huge, then the dropping time will be very long. We need to make it processed in parallel. In this patch, dropping blocks operation are processed in multiple threads, before dropping, each thread will select the blocks that to be dropped for itself.