File size: 5,322 Bytes
107f686 eca7c74 107f686 eca7c74 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 |
---
license: apache-2.0
language:
- zh
- en
pipeline_tag: question-answering
---
# Chinese-Alpaca-Plus-13B-GPTQ
This is GPTQ format quantised 4bit models of [Yiming Cui's Chinese-LLaMA-Alpaca 13B](https://github.com/ymcui/Chinese-LLaMA-Alpaca).
It is the result of quantising to 4bit using [GPTQ-for-LLaMa](https://github.com/qwopqwop200/GPTQ-for-LLaMa).
## Model Details
### Model Description
- **Developed by:** [ymcui (Yiming Cui)](https://github.com/ymcui)
- **Shared by:** Known Rabbit
- **Language(s) (NLP):** Chinese, English
- **License:** Apache 2.0
- **Finetuned from model:** LLaMA
The original Github project: [ymcui/Chinese-LLaMA-Alpaca: 中文LLaMA&Alpaca大语言模型+本地CPU/GPU部署 (Chinese LLaMA & Alpaca LLMs)](https://github.com/ymcui/Chinese-LLaMA-Alpaca)
> In order to promote the open research of large models in the Chinese NLP community, this project open sourced the Chinese LLaMA model and the Alpaca large model with fine-tuned instructions. Based on the original LLaMA, these models expand the Chinese vocabulary and use Chinese data for secondary pre-training, which further improves the basic semantic understanding of Chinese. At the same time, the Chinese Alpaca model further uses Chinese instruction data for fine-tuning, which significantly improves the model's ability to understand and execute instructions. For details, please refer to the technical report (Cui, Yang, and Yao, 2023).
### Model Sources
<!-- Provide the basic links for the model. -->
- **Repository:** https://github.com/ymcui/Chinese-LLaMA-Alpaca
- **Paper:** [[2304.08177] Efficient and Effective Text Encoding for Chinese LLaMA and Alpaca](https://arxiv.org/abs/2304.08177)
## Uses
<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
### Direct Use
<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
#### How to easily download and use this model in text-generation-webui
Open the text-generation-webui UI as normal.
1. Click the **Model tab**.
2. Under **Download custom model or LoRA**, enter `rabitt/Chinese-Alpaca-Plus-13B-GPTQ`.
3. Click **Download**.
4. Wait until it says it's finished downloading.
5. Click the **Refresh** icon next to **Model** in the top left.
6. In the **Model drop-down**: choose the model you just downloaded, `Chinese-Alpaca-Plus-13B-GPTQ`.
7. If you see an error like `Error no file named pytorch_model.bin ...` in the bottom right, ignore it - it's temporary.
8. Fill out the `GPTQ parameters` on the right: `Bits = 4`, `Groupsize = 128`, `model_type = Llama`
9. Click **Save settings for this model** in the top right.
10. Click **Reload the Model** in the top right.
11. Once it says it's loaded, click the **Text Generation tab** and enter a prompt!
## Training Details
### Training Procedure
1. Download models from the following links
* Original LLaMA: https://github.com/facebookresearch/llama/pull/73
* Chinese-LLaMA-Plus-13B
* [ziqingyang/chinese-llama-plus-lora-13b · Hugging Face](https://huggingface.co/ziqingyang/chinese-llama-plus-lora-13b)
* [chinese_llama_plus_lora_13b.zip_免费高速下载|百度网盘-分享无限制](https://pan.baidu.com/s/1VGpNlrLx5zHuNzLOcTG-xw?pwd=8cvd)
* Chinese-Alpaca-Plus-13B
* [ziqingyang/chinese-alpaca-plus-lora-13b · Hugging Face](https://huggingface.co/ziqingyang/chinese-alpaca-plus-lora-13b)
* [chinese_alpaca_plus_lora_13b.zip_免费高速下载|百度网盘-分享无限制](https://pan.baidu.com/s/1Mew4EjBlejWBBB6_WW6vig?pwd=mf5w)
2. Convert LLaMA to HuggingFace (HF) format with `convert_llama_weights_to_hf.py`
```bash
wget https://github.com/huggingface/transformers/raw/main/src/transformers/models/llama/convert_llama_weights_to_hf.py
PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python \
python convert_llama_weights_to_hf.py \
--input_dir ./llama \
--model_size 13B \
--output_dir ./llama-13b-hf
```
3. Merge `Chinese-LLaMA-Plus-13B` and `Chinese-Alpaca-Plus-13B` into LLaMA with `merge_llama_with_chinese_lora.py`
```bash
wget https://github.com/ymcui/Chinese-LLaMA-Alpaca/raw/main/scripts/merge_llama_with_chinese_lora.py
python merge_llama_with_chinese_lora.py \
--base_model ./llama-13b-hf \
--lora_model ./Chinese-LLaMA-Plus-LoRA-13B,./Chinese-Alpaca-Plus-LoRA-13B \
--output_type huggingface \
--output_dir ./Chinese-Alpaca-Plus-13B
```
4. Quantise the model with `GPTQ-for-LLaMa`
```bash
mkdir -p Chinese-Alpaca-Plus-13B-GPTQ
git clone https://github.com/qwopqwop200/GPTQ-for-LLaMa.git
cd GPTQ-for-LLaMa
# export CUDA_VISIBLE_DEVICES=0
python llama.py ../Chinese-Alpaca-Plus-13B c4 --wbits 4 --true-sequential --act-order --groupsize 128 --save_safetensors ../Chinese-Alpaca-Plus-13B-GPTQ/Chinese-Alpaca-Plus-13B-GPTQ-4bit-128g.safetensors
```
## Citation
**BibTeX:**
```tex
@article{chinese-llama-alpaca,
title={Efficient and Effective Text Encoding for Chinese LLaMA and Alpaca},
author={Cui, Yiming and Yang, Ziqing and Yao, Xin},
journal={arXiv preprint arXiv:2304.08177},
url={https://arxiv.org/abs/2304.08177},
year={2023}
}
```
|