TheBloke

/

vicuna-7B-v0-GPTQ

like 15

Hello,

I am using the TheBloke Runpod config, when I leave .safetensors I get stuff like:
USER: What time is it
ASSISTANT:�‑metÃmente SamGTzeti���� Dum Nueistoletr�Ð�Ã�Ã�ÃÄSpanÃÂÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃ

But when deleting the .safetensors and using the .pt I now get:
size mismatch for model.embed_tokens.weight: copying a param with shape torch.Size([32001, 4096]) from checkpoint, the shape in current model is torch.Size([32000, 4096]).

Thank you!