"Local variable ‘context_length’ referenced before assignment" when tokenizing a sentence

#39
by liliang - opened

The problem is as follows:

截屏2023-04-12 16.13.18.png

I guess 'context_length' should be computed before line 409 similar to line 394-397 in tokenization_chatglm.py?

Knowledge Engineering Group (KEG) & Data Mining at Tsinghua University org

Already fixed.

zxdu20 changed discussion status to closed

Sign up or log in to comment