Update README.md

README.md

@@ -1,3 +1,117 @@
 ---
 license: apache-2.0
 ---
+---
+license: apache-2.0
+tags:
+- text2text-generation
+pipeline_tag: text2text-generation
+language:
+- zh
+- en
+---
+
+# GPTQ-for-Bloom
+
+## Welcome
+If you find this model helpful, please *like* this model and star us on https://github.com/LianjiaTech/BELLE !
+
+## Model description
+8 bits quantization of [BELLE-7B-2M](https://huggingface.co/BelleGroup/BELLE-7B-2M) and [BELLE-7B-0.2M](https://huggingface.co/BelleGroup/BELLE-7B-0.2M) using [GPTQ](https://arxiv.org/abs/2210.17323)
+
+GPTQ is SOTA one-shot weight quantization method.
+
+The code of inference can be found in our Github project repository: https://github.com/LianjiaTech/BELLE/tree/main/gptq.
+
+Basically, 8-bit quantization and 128 groupsize are recommended.
+
+**This code is based on [GPTQ-for-LLaMa](https://github.com/qwopqwop200/GPTQ-for-LLaMa) for [Bloom](https://arxiv.org/pdf/2211.05100.pdf) model**
+
+## Model list
+
+| model name       |  file size | GPU memory usage |
+| -------------------------------------------------- |  ------------------- | ------------------ |
+|           base                 |          27G        |       ~28.2G         |
+|           bloom7b-2m-8bit-128g.pt                  |          9.7G        |       ~11.4G          |
+|           bloom7b-2m-4bit-128g.pt                  |          6.9G        |        ~8.4G          |
+|           bloom7b-0.2m-8bit-128g.pt                  |          9.7G        |       ~11.4G          |
+|           bloom7b-0.2m-4bit-128g.pt                  |          6.9G        |        ~8.4G          |
+
+## Limitations
+There still exists a few issues in the model trained on current base model and data:
+
+1. The model might generate factual errors when asked to follow instructions related to facts.
+
+2. Occasionally generates harmful responses since the model still struggles to identify potential harmful instructions.
+
+3. Needs improvements on reasoning and coding.
+
+Since the model still has its limitations, we require developers only use the open-sourced code, data, model and any other artifacts generated via this project for research purposes. Commercial use and other potential harmful use cases are not allowed.
+
+## Citation
+
+Please cite us when using our code, data or model.
+
+```
+@misc{BELLE,
+  author = {Yunjie Ji, Yong Deng, Yan Gong, Yiping Peng, Qiang Niu, Baochang Ma, Xiangang Li},
+  title = {BELLE: Bloom-Enhanced Large Language model Engine },
+  year = {2023},
+  publisher = {GitHub},
+  journal = {GitHub repository},
+  howpublished = {\url{https://github.com/LianjiaTech/BELLE}},
+}
+```
+
+Cite the original BLOOM, Stanford Alpaca and Self-Instruct papers as well!
+
+***
+
+# GPTQ-for-Bloom
+
+## 欢迎
+如果您觉得此模型对您有帮助，请like此模型并在https://github.com/LianjiaTech/BELLE 项目中star我们！
+
+## 模型描述
+对[BELLE-7B-2M](https://huggingface.co/BelleGroup/BELLE-7B-2M) and [BELLE-7B-0.2M](https://huggingface.co/BelleGroup/BELLE-7B-0.2M)进行8 bit（8位）量化。
+
+GPTQ是目前SOTA的one-shot权重量化方法。
+
+此模型的推理代码请见https://github.com/LianjiaTech/BELLE/tree/main/gptq .
+
+一般来说，推荐使用8-bit量化及groupsize = 128.
+
+**[Bloom](https://arxiv.org/pdf/2211.05100.pdf)模型使用[GPTQ](https://arxiv.org/abs/2210.17323)的推理代码基于[GPTQ-for-LLaMa](https://github.com/qwopqwop200/GPTQ-for-LLaMa)**
+
+## 模型列表
+
+| 模型名称       |  文件大小 | GPU显存占用 |
+| -------------------------------------------------- |  ------------------- | ------------------ |
+|           base                 |          27G        |       ~28.2G         |
+|           bloom7b-2m-4bit-128g.pt                  |          5.0G        |        ~8.0G          |
+
+
+## 局限性和使用限制
+基于当前数据和基础模型训练得到的SFT模型，在效果上仍存在以下问题：
+
+1. 在涉及事实性的指令上可能会产生违背事实的错误回答。
+
+2. 对于具备危害性的指令无法很好的鉴别，由此会产生危害性言论。
+
+3. 在一些涉及推理、代码等场景下模型的能力仍有待提高。
+
+基于以上模型局限性，我们要求开发者仅将我们开源的代码、数据、模型及后续用此项目生成的衍生物用于研究目的，不得用于商业，以及其他会对社会带来危害的用途。
+
+## 引用
+如果使用本项目的代码、数据或模型，请引用本项目。
+```
+@misc{BELLE,
+  author = {Yunjie Ji, Yong Deng, Yan Gong, Yiping Peng, Qiang Niu, Baochang Ma, Xiangang Li},
+  title = {BELLE: Bloom-Enhanced Large Language model Engine },
+  year = {2023},
+  publisher = {GitHub},
+  journal = {GitHub repository},
+  howpublished = {\url{https://github.com/LianjiaTech/BELLE}},
+}
+```
+也请同时引用原始的BLOOM论文、Stanford Alpaca和Self-Instruct论文。