Quantized model
#3
by
MaziyarPanahi
- opened
Thanks for sharing this model with the community, pretty interesting! I have quantized it in GGUF for people with low resources:
https://huggingface.co/MaziyarPanahi/zephyr-7b-gemma-v0.1-GGUF
@MaziyarPanahi llama_model_load: error loading model: create_tensor: tensor 'blk.0.attn_q.weight' has wrong shape; expected 3072, 3072, got 3072, 4096, 1, 1
Hi
@ReXommendation
seems the conversion script took the wrong parameters from the model which results in:
llama_model_load: error loading model: create_tensor: tensor 'blk.0.attn_q.weight' has wrong shape; expected 3072, 3072, got 3072, 4096, 1, 1
Thanks for reporting this. I'll create an issue on Llama.cpp and hopefully it gets resolved