Added various bpw branches
Browse files
README.md
CHANGED
|
@@ -16,6 +16,17 @@ You are currently at the [main](/kennylam/Breeze-7B-Instruct-64k-v0_1-exl2/tree/
|
|
| 16 |
這裡是main branch, 只提供EvLlamaV2量化時所用到的[measurement.json](measurement.json)檔案。
|
| 17 |
|
| 18 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 19 |
## Citation
|
| 20 |
|
| 21 |
```
|
|
|
|
| 16 |
這裡是main branch, 只提供EvLlamaV2量化時所用到的[measurement.json](measurement.json)檔案。
|
| 17 |
|
| 18 |
|
| 19 |
+
[8.0bpw-h8](/kennylam/Breeze-7B-Instruct-64k-v0_1-exl2/tree/8.0bpw-h8) 8 bits per weight.
|
| 20 |
+
|
| 21 |
+
[6.0bpw-h6](/kennylam/Breeze-7B-Instruct-64k-v0_1-exl2/tree/6.0bpw-h6) 6 bits per weight.
|
| 22 |
+
|
| 23 |
+
[5.0bpw-h6](/kennylam/Breeze-7B-Instruct-64k-v0_1-exl2/tree/5.0bpw-h6) 5 bits per weight.
|
| 24 |
+
|
| 25 |
+
[4.0bpw-h6](/kennylam/Breeze-7B-Instruct-64k-v0_1-exl2/tree/4.0bpw-h6) 4 bits per weight.
|
| 26 |
+
|
| 27 |
+
[3.0bpw-h6](/kennylam/Breeze-7B-Instruct-64k-v0_1-exl2/tree/3.0bpw-h6) 3 bits per weight.
|
| 28 |
+
|
| 29 |
+
|
| 30 |
## Citation
|
| 31 |
|
| 32 |
```
|