RuntimeError: shape '[4, 226, 24576]' is invalid for input of size 9256960
#26
by
linkai-dl
- opened
Getting this error with 70B, but I can get both 13B and 7B work with GPTQ.
File /usr/local/lib/python3.10/site-packages/auto_gptq/nn_modules/qlinear_old.py:249, in QuantLinear.forward(self, x)
246 weight = weight.reshape(weight.shape[0] * weight.shape[1], weight.shape[2])
248 out = torch.matmul(x.half(), weight)
--> 249 out = out.half().reshape(out_shape)
250 out = out + self.bias if self.bias is not None else out
251 return out
RuntimeError: shape '[4, 226, 24576]' is invalid for input of size 9256960
Make sure you've updated to latest AutoGPTQ (currently 0.3.2) and have Transformers 4.31.0.
If you are already on the latest version, please double check the file downloaded OK.
wow what a speed! thanks, will try upgrading.
linkai-dl
changed discussion status to
closed
I got every package updated but got another error:
ValueError: not enough values to unpack (expected 3, got 2)
I am sure the files were correct.
linkai-dl
changed discussion status to
open
I also met this problem, transformer 4.31 didn't solve it