load error in transformers?
#2
by
bdambrosio
- opened
ValueError: Wrong index found for [PAD]: should be 32001 but found 32000.
I'll admit I didn't go far in trying to debug this, always find token dictionary size problems confusing.
Do I need to add a pad token to tokenizer or something?
Please try downloading again. I just fixed tokenizer.json