No token for "the"?
#2
by
FremyCompany
- opened
I think there might be something strange going on with the tokenizer, because it does not have a token for "Ġthe" despite having a token for "ĠThe": 409.
I think there might be something strange going on with the tokenizer, because it does not have a token for "Ġthe" despite having a token for "ĠThe": 409.