Return immediately on invalid character

Fixes #2458