Copy the Set into a mutable HashSet