---
license: llama2
---

GPTQ 4-bit actor order version that works in textgen-webui with exllamav2

Generated by using scripts from https://gitee.com/yhyu13/llama_-tools

Original weight : https://huggingface.co/Xwin-LM/Xwin-Math-7B-V1.0

---

| Branch | Bits | GS | Act Order | Damp % | GPTQ Dataset | Seq Len | Size | ExLlama | Desc |
| ------ | ---- | -- | --------- | ------ | ------------ | ------- | ---- | ------- | ---- |
| [main](https://huggingface.co/Yhyu13/Xwin-Math-7B-V1.0-GPTQ-4bit/tree/main) | 4 | 128 | Yes | 0.1 | [code](wikitext2) | 4096 | 4.4GB | Yes | 4-bit, with Act Order. 128 group size | 

---

Here is some testing done in textgen-webui, I was using Q&A from this dataset https://huggingface.co/datasets/TIGER-Lab/MathInstruct

Basic arithmatic, the answer (A) is correct
![Alt text](img/textgen-xinmath.png)


Does not follow the instruction to write python code. And hulluciate an answer not in exists in options
![Alt text](img/textgen-xinmath2.png)

Answer is correct, but hulluciate with a not existing option
![Alt text](img/textgen-xinmath3.png)