RuntimeError: probability tensor contains either `inf`, `nan` or element < 0
env:
python 3.11.7
pytorch 2.2.1
transformers 4.37.2
auto_gptq 0.7.1
Loading checkpoint shards: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:06<00:00, 2.28s/it]
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
Traceback (most recent call last):
File "/root/model/test7b.py", line 23, in
generated_ids = model.generate(
^^^^^^^^^^^^^^^
File "/root/.pyenv/versions/env311/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/root/.pyenv/versions/env311/lib/python3.11/site-packages/transformers/generation/utils.py", line 1525, in generate
return self.sample(
^^^^^^^^^^^^
File "/root/.pyenv/versions/env311/lib/python3.11/site-packages/transformers/generation/utils.py", line 2658, in sample
next_tokens = torch.multinomial(probs, num_samples=1).squeeze(1)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: probability tensor contains either inf, nan or element < 0
solved: pytorch2.2.1 -> pytorch2.1.2