请问有gptq或者gguf版本吗

#4
by Lenosey - opened
Beijing Academy of Artificial Intelligence org

To our surprise, loading the Aquila2-34B model with bitsandbytes INT4 shows that it can retain 99.3% of its performance.

Sign up or log in to comment