@mengxr Won't an individual partition fit comfortably in memory?  The HashSet will be considerably smaller than the set of ratings in the partition.