[SPARK-1201] Do not fully materialize partitions for MEMORY_*_SER

The deserialized version of a partition may occupy much more space than the serialized version. Therefore, if a partition is to be cached with `StorageLevel.MEMORY_*_SER`, we don't need to fully unroll it into an `ArrayBuffer`, but instead we can unroll it into a potentially much smaller `ByteBuffer`. This may save us from OOMs in this case.
Yeah, this comment already existed and it wasn't exactly clear to me why we needed the lock in the first place. However, assuming the unlikely case that two threads do work on the same partition, it is technically possible for a race condition to occur where the block is kicked out of memory immediately after the other thread released the lock. In this case when we try to `get` the block we will return `None`.