Changing to mapPartition based calculations, to avoid reloading data.