ValueError: Non-consecutive added token '<|endoftext|>' found. Should have index 50258 but has index 50257 in saved vocabulary.

#7
by rabintang - opened

ValueError: Non-consecutive added token '<|endoftext|>' found. Should have index 50258 but has index 50257 in saved vocabulary.
加载模型出现如上错误,请问是什么原因

update transformers 到4.35以上

Sign up or log in to comment