Hugging Face's logo

CohereForAI
/
c4ai-command-r-plus

Text Generation
Transformers
Safetensors
10 languages
cohere
conversational
Inference Endpoints
text-generation-inference
Model card Files Community
46

GPUs Requirement For Inference

#42
by Aillian - opened
Discussion
Aillian

What is the GPUs requirement for inference on this model?

Kaz-K
•
edited 18 days ago

I wrote this down by mistake.

superMe

Any answer ?

Aillian

Referring https://huggingface.co/CohereForAI/c4ai-command-r-plus-4bit/discussions/7#6634808be155506766967f9a

BigHuggyD

I am running exl2 5.0bpw with 64k context at this very moment on 90.72 GB

Sign up or log in to comment