RuntimeError: The size of tensor a (4096) must match the size of tensor b (1479) at non-singleton dimension 3
#1
by
Tonight223
- opened
Any one know how to fix this? Running model, set seq length 4096. After set the seq length as tensor b (1479) it went ok again but one message later it shows similiar error again. And the offical model card said max seq length is 4096, what's wrong? And I find out if I reload the model with 4096 len every time before sending the message it will be ok. I tried other AWQ model they have similiar issues.
