Hugging Face's logo

BigHuggyD
/
alpindale_magnum-72b-v1_exl2_4.0bpw_h8

Text Generation
Transformers
Safetensors
English
Chinese
qwen2
chat
conversational
text-generation-inference
Inference Endpoints
4-bit precision
exl2
Model card Files Community
2
New discussion

what quant should I use to use this with a single 24GB video card (PC) (4090 card)?

1
#2 opened 5 months ago by clevnumb