Fix error handling for invalid numbers

This solves a number of issues exposed by the addition of test
for invalid numeric inputs:

- Hex and octal numbers with sign are now reported as errors.
- Error messages are properly formatted.
- Exponents and dots in hex and octal numbers are reported as
  errors.
- Hexadecimal digits are now reported as errors in floating
  point and integer numbers.