thank you for the great work, can this be converted to gguf?
#1
by
asidaddy
- opened
Hi guys,
in order to use in an existing pipeline, we need this in gguf, regular llamacpp doesn't work for this case. can you suggest what do to?
raise ValueError(f"Can not map tensor {name!r}")
ValueError: Can not map tensor 'model.layers.0.mlp.down_proj.weight.absmax'
Hi guys,
in order to use in an existing pipeline, we need this in gguf, regular llamacpp doesn't work for this case. can you suggest what do to?
raise ValueError(f"Can not map tensor {name!r}")
ValueError: Can not map tensor 'model.layers.0.mlp.down_proj.weight.absmax'
You can directly use our GGUF instead and use Q4: https://huggingface.co/unsloth/DeepSeek-R1-Distill-Llama-70B-GGUF