Remove unnecessary require of Set.

It looks like this was introduced in 91b78211d086010ee34a714715c4d49a8d5a1f19 but isn't used anymore.