Because the RDD object isnt available here (already garbage collected). Yes, I am aware that this is a duplicate code path. Can you think of a better place (that is, not in RDD class) where we can move this functionality?