kennylam commited on
Commit
455397e
1 Parent(s): b74524c

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +20 -0
README.md ADDED
@@ -0,0 +1,20 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ## CantoneseLLM-6B-preview202402 with ExLlamaV2 Quantization
2
+ 哩個係用 [/hon9kon9ize/CantoneseLLM-6B-preview202402](https://huggingface.co/hon9kon9ize/CantoneseLLM-6B-preview202402) 生成嘅exl2量化模型。<br>
3
+ This is a quantizated model from [/hon9kon9ize/CantoneseLLM-6B-preview202402](https://huggingface.co/hon9kon9ize/CantoneseLLM-6B-preview202402) in exl2 format.<br>
4
+ 這是一個由 [/hon9kon9ize/CantoneseLLM-6B-preview202402](https://huggingface.co/hon9kon9ize/CantoneseLLM-6B-preview202402) 生成的exl2量化模型。
5
+
6
+
7
+ 哩度係main branch, 只係放EvLlamaV2量化果陣用到嘅[measurement.json](measurement.json)檔案,請響下面揀量化程度。<br>
8
+ You are currently at the [main](https://huggingface.co/kennylam/CantoneseLLM-6B-preview202402-exl2/tree/main) branch, which provides only [measurement.json](measurement.json) used in the ExLlamaV2 quantization. Please take a look of your choices in following table of branches.<br>
9
+ 這裡是main branch, 只提供EvLlamaV2量化時所用到的[measurement.json](measurement.json)檔案,請在下面選擇量化程度。。
10
+
11
+
12
+ [8.0bpw-h8](/kennylam/CantoneseLLM-6B-preview202402-exl2/tree/8.0bpw-h8) 8 bits per weight.
13
+
14
+ [6.0bpw-h6](/kennylam/CantoneseLLM-6B-preview202402-exl2/tree/6.0bpw-h6) 6 bits per weight.
15
+
16
+ [5.0bpw-h6](/kennylam/CantoneseLLM-6B-preview202402-exl2/tree/5.0bpw-h6) 4 bits per weight.
17
+
18
+ [4.0bpw-h6](/kennylam/CantoneseLLM-6B-preview202402-exl2/tree/4.0bpw-h6) 4 bits per weight.
19
+
20
+ [3.0bpw-h6](/kennylam/CantoneseLLM-6B-preview202402-exl2/tree/3.0bpw-h6) 3 bits per weight.