Common: Fix AsciiToHex returning true on overflow values

We should be checking errno against ERANGE.
Merge pull request #785 from lioncash/string

Common: Fix AsciiToHex returning true on overflow values