CohereForAI

/

c4ai-command-r-v01

like 997

Yes i second this, we need gguf

Looking for the gguf too.

In the meantime, how could I test it? With Ollama?

read ollama docs on how to create new model

I imagine that @TheBloke is already firing up the stoves.

Apparently, it won't work with Ollama right now (from their Discord).

I imagine that @TheBloke is already firing up the stoves.

He actually stopped uploading GGUF models like 1 month ago.

Meanwhile I am trying to make it work with HQQ

I was able to convert the safetensors to a GGUF model. I'm still working on adding inference support to llama.cpp.
See PR: https://github.com/ggerganov/llama.cpp/pull/6033

where is the Gguff format this model ?

Starting to add the GGUF files here: https://huggingface.co/andrewcanis/c4ai-command-r-v01-GGUF

Starting to add the GGUF files here: https://huggingface.co/andrewcanis/c4ai-command-r-v01-GGUF

Great work Andrew! Both on the GGUF models and specially on the PR you made in Llama.cpp.
Thank you

Is there GPTQ or AWQ version?