Remove on-by-default warning from -Wmost.

-Wint-to-pointer-cast, added to match GCC's warning by the same name, doesn't
need to be in any other groups (as it isn't in any groups in GCC either).

Found in post-commit review by Ted Kremenek.