Changed tokenization behaviour on error.
If the error is only a tokenization error and not an unexpected error the tok_seq object is creted with the tokens before the error.