RuntimeError: probability tensor contains either `inf`, `nan` or element < 0

#7
by Satya93 - opened

I get this error using the newest transformers 4.31.0 on my RTX 4090. I am loading the model like this:

tokenizer = AutoTokenizer.from_pretrained(model_dir, use_fast=False)
model = AutoModelForCausalLM.from_pretrained(model_dir,torch_dtype=torch.float16, low_cpu_mem_usage=True, device_map="auto")

Any ideas?

Sign up or log in to comment