Update README.md
Browse files
README.md
CHANGED
@@ -14,15 +14,15 @@ language:
|
|
14 |
If you find this model helpful, please *like* this model and star us on https://github.com/LianjiaTech/BELLE !
|
15 |
|
16 |
## Model description
|
17 |
-
8 bits quantization of [
|
18 |
|
19 |
GPTQ is SOTA one-shot weight quantization method.
|
20 |
|
21 |
-
The code of inference can be found in our Github project repository: https://github.com/LianjiaTech/BELLE/gptq.
|
22 |
|
23 |
Basically, 8-bit quantization and 128 groupsize are recommended.
|
24 |
|
25 |
-
**This code is based on [GPTQ-for-LLaMa](https://github.com/qwopqwop200/GPTQ-for-LLaMa)**
|
26 |
|
27 |
## Model list
|
28 |
|
@@ -34,6 +34,17 @@ Basically, 8-bit quantization and 128 groupsize are recommended.
|
|
34 |
| bloom7b-0.2m-8bit-128g.pt | 9.7G | ~11.4G |
|
35 |
| bloom7b-0.2m-4bit-128g.pt | 6.9G | ~8.4G |
|
36 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
37 |
## Citation
|
38 |
|
39 |
Please cite us when using our code, data or model.
|
@@ -59,15 +70,15 @@ Cite the original BLOOM, Stanford Alpaca and Self-Instruct papers as well!
|
|
59 |
如果您觉得此模型对您有帮助,请like此模型并在https://github.com/LianjiaTech/BELLE 项目中star我们!
|
60 |
|
61 |
## 模型描述
|
62 |
-
对[
|
63 |
|
64 |
GPTQ是目前SOTA的one-shot权重量化方法。
|
65 |
|
66 |
-
此模型的推理代码请见https://github.com/LianjiaTech/BELLE/gptq .
|
67 |
|
68 |
一般来说,推荐使用8-bit量化及groupsize = 128.
|
69 |
|
70 |
-
|
71 |
|
72 |
## 模型列表
|
73 |
|
@@ -79,6 +90,17 @@ GPTQ是目前SOTA的one-shot权重量化方法。
|
|
79 |
| bloom7b-0.2m-8bit-128g.pt | 9.7G | ~11.4G |
|
80 |
| bloom7b-0.2m-4bit-128g.pt | 6.9G | ~8.4G |
|
81 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
82 |
## 引用
|
83 |
如果使用本项目的代码、数据或模型,请引用本项目。
|
84 |
```
|
|
|
14 |
If you find this model helpful, please *like* this model and star us on https://github.com/LianjiaTech/BELLE !
|
15 |
|
16 |
## Model description
|
17 |
+
8 bits quantization of [BELLE-7B-2M](https://huggingface.co/BelleGroup/BELLE-7B-2M) and [BELLE-7B-0.2M](https://huggingface.co/BelleGroup/BELLE-7B-0.2M) using [GPTQ](https://arxiv.org/abs/2210.17323)
|
18 |
|
19 |
GPTQ is SOTA one-shot weight quantization method.
|
20 |
|
21 |
+
The code of inference can be found in our Github project repository: https://github.com/LianjiaTech/BELLE/tree/main/gptq.
|
22 |
|
23 |
Basically, 8-bit quantization and 128 groupsize are recommended.
|
24 |
|
25 |
+
**This code is based on [GPTQ-for-LLaMa](https://github.com/qwopqwop200/GPTQ-for-LLaMa) for [Bloom](https://arxiv.org/pdf/2211.05100.pdf) model**
|
26 |
|
27 |
## Model list
|
28 |
|
|
|
34 |
| bloom7b-0.2m-8bit-128g.pt | 9.7G | ~11.4G |
|
35 |
| bloom7b-0.2m-4bit-128g.pt | 6.9G | ~8.4G |
|
36 |
|
37 |
+
## Limitations
|
38 |
+
There still exists a few issues in the model trained on current base model and data:
|
39 |
+
|
40 |
+
1. The model might generate factual errors when asked to follow instructions related to facts.
|
41 |
+
|
42 |
+
2. Occasionally generates harmful responses since the model still struggles to identify potential harmful instructions.
|
43 |
+
|
44 |
+
3. Needs improvements on reasoning and coding.
|
45 |
+
|
46 |
+
Since the model still has its limitations, we require developers only use the open-sourced code, data, model and any other artifacts generated via this project for research purposes. Commercial use and other potential harmful use cases are not allowed.
|
47 |
+
|
48 |
## Citation
|
49 |
|
50 |
Please cite us when using our code, data or model.
|
|
|
70 |
如果您觉得此模型对您有帮助,请like此模型并在https://github.com/LianjiaTech/BELLE 项目中star我们!
|
71 |
|
72 |
## 模型描述
|
73 |
+
对[BELLE-7B-2M](https://huggingface.co/BelleGroup/BELLE-7B-2M) and [BELLE-7B-0.2M](https://huggingface.co/BelleGroup/BELLE-7B-0.2M)进行8 bit(8位)量化。
|
74 |
|
75 |
GPTQ是目前SOTA的one-shot权重量化方法。
|
76 |
|
77 |
+
此模型的推理代码请见https://github.com/LianjiaTech/BELLE/tree/main/gptq .
|
78 |
|
79 |
一般来说,推荐使用8-bit量化及groupsize = 128.
|
80 |
|
81 |
+
**[Bloom](https://arxiv.org/pdf/2211.05100.pdf)模型使用[GPTQ](https://arxiv.org/abs/2210.17323)的推理代码基于[GPTQ-for-LLaMa](https://github.com/qwopqwop200/GPTQ-for-LLaMa)**
|
82 |
|
83 |
## 模型列表
|
84 |
|
|
|
90 |
| bloom7b-0.2m-8bit-128g.pt | 9.7G | ~11.4G |
|
91 |
| bloom7b-0.2m-4bit-128g.pt | 6.9G | ~8.4G |
|
92 |
|
93 |
+
## 局限性和使用限制
|
94 |
+
基于当前数据和基础模型训练得到的SFT模型,在效果上仍存在以下问题:
|
95 |
+
|
96 |
+
1. 在涉及事实性的指令上可能会产生违背事实的错误回答。
|
97 |
+
|
98 |
+
2. 对于具备危害性的指令无法很好的鉴别,由此会产生危害性言论。
|
99 |
+
|
100 |
+
3. 在一些涉及推理、代码等场景下模型的能力仍有待提高。
|
101 |
+
|
102 |
+
基于以上模型局限性,我们要求开发者仅将我们开源的代码、数据、模型及后续用此项目生成的衍生物用于研究目的,不得用于商业,以及其他会对社会带来危害的用途。
|
103 |
+
|
104 |
## 引用
|
105 |
如果使用本项目的代码、数据或模型,请引用本项目。
|
106 |
```
|