[SPARK-4027][Streaming] HDFSBasedBlockRDD to read received either from BlockManager or WAL in HDFS

As part of the initiative of preventing data loss on streaming driver failure, this sub-task implements a BlockRDD that is backed by HDFS. This BlockRDD can either read data from the Spark's BlockManager, or read the data from file-segments in write ahead log in HDFS. 

Most of this code has been written by @harishreedharan

you should explain more clearly what this test function does.  it is long enough that it is no longer obvious, and your comment doesn't really address that.