Create README.md
#1
by
kennylam
- opened
README.md
ADDED
|
@@ -0,0 +1,20 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
## CantoneseLLM-6B-preview202402 with ExLlamaV2 Quantization
|
| 2 |
+
哩個係用 [/hon9kon9ize/CantoneseLLM-6B-preview202402](https://huggingface.co/hon9kon9ize/CantoneseLLM-6B-preview202402) 生成嘅exl2量化模型。<br>
|
| 3 |
+
This is a quantizated model from [/hon9kon9ize/CantoneseLLM-6B-preview202402](https://huggingface.co/hon9kon9ize/CantoneseLLM-6B-preview202402) in exl2 format.<br>
|
| 4 |
+
這是一個由 [/hon9kon9ize/CantoneseLLM-6B-preview202402](https://huggingface.co/hon9kon9ize/CantoneseLLM-6B-preview202402) 生成的exl2量化模型。
|
| 5 |
+
|
| 6 |
+
|
| 7 |
+
哩度係main branch, 只係放EvLlamaV2量化果陣用到嘅[measurement.json](measurement.json)檔案，請響下面揀量化程度。<br>
|
| 8 |
+
You are currently at the [main](https://huggingface.co/kennylam/CantoneseLLM-6B-preview202402-exl2/tree/main) branch, which provides only [measurement.json](measurement.json) used in the ExLlamaV2 quantization. Please take a look of your choices in following table of branches.<br>
|
| 9 |
+
這裡是main branch, 只提供EvLlamaV2量化時所用到的[measurement.json](measurement.json)檔案，請在下面選擇量化程度。。
|
| 10 |
+
|
| 11 |
+
|
| 12 |
+
[8.0bpw-h8](/kennylam/CantoneseLLM-6B-preview202402-exl2/tree/8.0bpw-h8) 8 bits per weight.
|
| 13 |
+
|
| 14 |
+
[6.0bpw-h6](/kennylam/CantoneseLLM-6B-preview202402-exl2/tree/6.0bpw-h6) 6 bits per weight.
|
| 15 |
+
|
| 16 |
+
[5.0bpw-h6](/kennylam/CantoneseLLM-6B-preview202402-exl2/tree/5.0bpw-h6) 4 bits per weight.
|
| 17 |
+
|
| 18 |
+
[4.0bpw-h6](/kennylam/CantoneseLLM-6B-preview202402-exl2/tree/4.0bpw-h6) 4 bits per weight.
|
| 19 |
+
|
| 20 |
+
[3.0bpw-h6](/kennylam/CantoneseLLM-6B-preview202402-exl2/tree/3.0bpw-h6) 3 bits per weight.
|