Remove unnecessary test in CharTypeOf

Very minor code-gnoming.

The idea is that integrals are implicitly castable to char types, so we test them explicitly, to detect them and turn them down.

Checking for `long`/`ulong` is not necessary though, since these aren't implicitly character cast-able to begin with.