apache/incubator-spark — pull request #167 by xiajunluan (2013-11-14 03:53:37+00:00)
Title: [JIRA-942]CacheManager/BlockManager handle generating iterators p...

In this PR, I manage to resolve this bug to avoid OOM exception when user only persists big RDD(beyond free memory size) with DISK_ONLY option, I will unroll the Iterator to ArrayBuffer only for MEMORY_ONLY or MEMORY_AND_DISK options, and deliver iterator directly to OutputStream for DISK_ONLY option.
Detail JIRA info:https://spark-project.atlassian.net/browse/SPARK-942