File size: 1,624 Bytes
9bf7178
 
 
 
 
 
 
 
 
 
 
 
 
c208173
 
 
9bf7178
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
c46a581
 
9bf7178
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
fc25351
9bf7178
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
---
title: chinese-alpaca-33b-merged
emoji: 📚
colorFrom: gray
colorTo: red
sdk: gradio
sdk_version: 3.23.0
app_file: app.py
pinned: false
---

加入中文词表并继续预训练中文Embedding,并在此基础上继续使用指令数据集finetuning,得到的中文Alpaca-33B模型。

模型转换用到的相关base及lora模型如下:
- base-model: elinas/llama-30b-hf-transformers-4.29
- lora-model: ziqingyang/chinese-alpaca-lora-33b

详情可参考:https://github.com/ymcui/Chinese-LLaMA-Alpaca/releases/tag/v4.0


### 使用方法参考
1. 安装模块包
```bash
pip install sentencepiece
pip install transformers>=4.28.0
```

2. 生成文本
```python
import torch
import transformers
from transformers import LlamaTokenizer, LlamaForCausalLM

def generate_prompt(text):
    return f"""Below is an instruction that describes a task. Write a response that appropriately completes the request.

### Instruction:
{text}

### Response:"""


tokenizer = LlamaTokenizer.from_pretrained('minlik/chinese-alpaca-33b-merged')
model = LlamaForCausalLM.from_pretrained('minlik/chinese-alpaca-33b-merged').half().to('cuda')
model.eval()

text = '第一个登上月球的人是谁?'
prompt = generate_prompt(text)
input_ids = tokenizer.encode(prompt, return_tensors='pt').to('cuda')


with torch.no_grad():
    output_ids = model.generate(
        input_ids=input_ids,
        max_new_tokens=128,
        temperature=1,
        top_k=40,
        top_p=0.9,
        repetition_penalty=1.15
    ).cuda()
output = tokenizer.decode(output_ids[0], skip_special_tokens=True)
print(output.replace(prompt, '').strip())
```