consider string size in hex2dec transformation

Reported by gergelypeli on GitHub
Closes #296

(cherry picked from commit 9b79b1bca1fc7fcfac751275de045e08728ddd00)