Text Generation
Transformers
PyTorch
Chinese
English
llama
text-generation-inference

有办法量化吗

#1
by zxgov - opened

可以的。
我使用AutoGPTQ实现了4-bits量化,可参考:baichuan-vicuna-chinese-7b-gptq
AutoGPTQ: https://github.com/PanQiWei/AutoGPTQ

Sign up or log in to comment