Wouldn't it be cheaper to remove the elements from one of the sets instead of checking for existence and then storing in a new set?