Hmm, can we avoid having to build a new Set just to drop it from the old? foldLeft?