fix invalid hex number detection also