maddes8cht

/

acrastt-Marx-3B-V3-gguf

like 1

I often get error from ctransformers when importing some GGUF models. Here is the code example

from ctransformers import AutoModelForCausalLM

llm = AutoModelForCausalLM.from_pretrained("maddes8cht/acrastt-Marx-3B-V3-gguf", model_file="maddes8cht/acrastt-Marx-3B-V3-gguf", model_type='stablelm')
print(llm("AI is going to"))

It actually downloaded the .gguf file from repo, but returned this error message

RuntimeError: Failed to create LLM 'stablelm' from '/root/.cache/huggingface/hub/models--

I got the same error with TheBloke/Marx-3B-v3-GGUF model

I've tried using different model_type such as mistral and llama, any idea?