Edit model card

GPTQ 4-bit actor order version that works in textgen-webui with exllamav2

Generated by using scripts from https://gitee.com/yhyu13/llama_-tools

Original weight : https://huggingface.co/Xwin-LM/Xwin-Math-7B-V1.0

Branch Bits GS Act Order Damp % GPTQ Dataset Seq Len Size ExLlama Desc
main 4 128 Yes 0.1 code 4096 4.4GB Yes 4-bit, with Act Order. 128 group size

Here is some testing done in textgen-webui, I was using Q&A from this dataset https://huggingface.co/datasets/TIGER-Lab/MathInstruct

Basic arithmatic, the answer (A) is correct Alt text

Does not follow the instruction to write python code. And hulluciate an answer not in exists in options Alt text

Answer is correct, but hulluciate with a not existing option Alt text

Downloads last month