keyfan
/

Qwen-72B-Chat-2bit

Text Generation

Model card Files Files and versions Community

keyfan commited on Jan 7

Commit

e3beafa

•

1 Parent(s): c7dfd64

Update README.md

Files changed (1) hide show

README.md +2 -2

README.md CHANGED Viewed

@@ -6,7 +6,7 @@ license_link: https://huggingface.co/Qwen/Qwen-72B-Chat/blob/main/LICENSE
 This is 2-bit quantization of [Qwen/Qwen-72B-Chat](https://huggingface.co/Qwen/Qwen-72B-Chat) using [QuIP#](https://cornell-relaxml.github.io/quip-sharp/)
-Random samples from C4 are used as calibration data. I'm not sure if it will have negative effect on Chinese tasks.
 ## Model loading
 Please follow the instruction of [QuIP-for-all](https://github.com/chu-tianxiang/QuIP-for-all) for usage.
@@ -17,7 +17,7 @@ As an alternative, you can use [vLLM branch](https://github.com/chu-tianxiang/vl
 Measured at Wikitext with 4096 context length
 | fp16 | 2-bit |
 | ------- | ------- |
-| 5.8438  | 6.9492  |
 ## Speed

 This is 2-bit quantization of [Qwen/Qwen-72B-Chat](https://huggingface.co/Qwen/Qwen-72B-Chat) using [QuIP#](https://cornell-relaxml.github.io/quip-sharp/)
+Random samples from C4 and [SkyPile](https://huggingface.co/datasets/Skywork/SkyPile-150B) are used as calibration data.
 ## Model loading
 Please follow the instruction of [QuIP-for-all](https://github.com/chu-tianxiang/QuIP-for-all) for usage.
 Measured at Wikitext with 4096 context length
 | fp16 | 2-bit |
 | ------- | ------- |
+| 5.8438  | 7.3047  |
 ## Speed