Remove unnecessary join
Revert to simple flatMap instead of mapPartitions