tokenizer: re.sre_compile.error doesn't necessary exists

Regex lib might be regex or default re, and we don't really need to know what
the exception is, so let's just expect Exception.
tokenizer: Add EOP token if needed