File size: 1,398 Bytes
470b62f 0ec3eb9 0159363 470b62f 0ef8e4b 470b62f 4061708 470b62f 6ecb1e1 470b62f 80df225 470b62f |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 |
---
license: apache-2.0
language:
- zh
- en
library_name: mlx
base_model:
- Qwen/Qwen2-7B-Instruct
tags:
- macbook
---
# BiliBot
b友风格聊天机器人
+ 基础模型: Qwen2-7B
+ 数据来源: [https://github.com/linyiLYi/bilibot/tree/main/data](https://github.com/linyiLYi/bilibot/tree/main/data)
+ 量化: 4bit
+ 推荐配置: 16G内存及以上的M系芯片Macbook
> 由于是MLX格式模型，首先需要安装 mlx-lm 包
```bash
pip install mlx-lm
```
下面是一个示例，用户可随意提问
```python
import time
from mlx_lm import load, generate
model, tokenizer = load('Kadins/BiliBot-7B-Q', tokenizer_config={"eos_token": "<|im_end|>"})
# Template content
template = """
<|im_start|>system
You are a helpful assistant<|im_end|>
<|im_start|>user
你是一位B站老用户，请你对以下问题给出简短、机智的回答：
{usr_msg}<|im_end|>
<|im_start|>assistant
"""
while True:
usr_msg = input("用户: ") # Get user message from terminal
if usr_msg.lower() == 'quit()': # Allows the user to exit the loop
break
prompt = template.replace("{usr_msg}", usr_msg)
time_ckpt = time.time()
response = generate(
model,
tokenizer,
prompt=prompt,
temp=0.3,
max_tokens=500,
verbose=False
)
print("%s: %s (Time %d ms)\n" % ("回答", response, (time.time() - time_ckpt) * 1000))
``` |