Hugging Face's logo

shenzhi-wang
/
Llama3-70B-Chinese-Chat

Text Generation
Transformers
Safetensors
English
Chinese
llama
llama-factory
orpo
conversational
Inference Endpoints
text-generation-inference
Model card Files Community
10

请问能否提供EXL2量化版本?

#7
by Orion-zhen - opened
Discussion
Orion-zhen

EXL2相较于GPTQ和AWQ, 有较低的模型和上下文显存占用, 更适合在消费端部署. 请问是否能提供EXL2量化版本? 例如4.0bpw

Orion-zhen

自己量化了一个: Orion-zhen/Llama3-70B-Chinese-Chat-4.0bpw-exl2

Sign up or log in to comment