File size: 2,738 Bytes
7d23211 1b12f01 7d23211 1b12f01 7d23211 1b12f01 b351e77 1b12f01 b351e77 1b12f01 b351e77 1b12f01 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 |
# NanoTranslator-immersive_translate-0.5B
[English](README.md) | 简体中文
## Introduction
NanoTranslator-immersive_translate-0.5B 是由 [Qwen2-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2-0.5B-Instruct) 在 [BiST](https://huggingface.co/datasets/Mxode/BiST) 和 [wmt19](https://huggingface.co/datasets/wmt/wmt19) 数据集上训练得到的专门用于**中英双语**的翻译模型。
此模型遵循[沉浸式翻译](https://immersivetranslate.com/)（Immersive Translate）的 prompt 格式进行训练，可以通过 vllm、lmdeploy 等方式部署为 OpenAI 格式接口，从而完成调用。
## How to use
下面是一个用 transformers 调用的方式，prompt 遵循沉浸式翻译以保持最佳效果。
```python
import torch
from typing import Literal
from transformers import AutoModelForCausalLM, AutoTokenizer
model_path = 'Mxode/NanoTranslator-immersive_translate-0.5B'
model = AutoModelForCausalLM.from_pretrained(model_path).to('cuda:0', torch.bfloat16)
tokenizer = AutoTokenizer.from_pretrained(model_path)
def translate(
text: str,
to: Literal["Chinese", "English"] = "Chinese",
**kwargs
):
generation_args = dict(
max_new_tokens = kwargs.pop("max_new_tokens", 512),
do_sample = kwargs.pop("do_sample", True),
temperature = kwargs.pop("temperature", 0.55),
top_p = kwargs.pop("top_p", 0.8),
top_k = kwargs.pop("top_k", 40),
**kwargs
)
prompt = """Translate the following source text to {to}. Output translation directly without any additional text.
Source Text: {text}
Translated Text:"""
messages = [
{"role": "system", "content": "You are a professional, authentic machine translation engine."},
{"role": "user", "content": prompt.format(to=to, text=text)}
]
inputs = tokenizer.apply_chat_template(
messages,
tokenize=False,
add_generation_prompt=True
)
model_inputs = tokenizer([inputs], return_tensors="pt").to(model.device)
generated_ids = model.generate(model_inputs.input_ids, **generation_args)
generated_ids = [
output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
]
response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
return response
text = "After a long day at work, I love to unwind by cooking a nice dinner and watching my favorite TV series. It really helps me relax and recharge for the next day."
response = translate(text=text, to='Chinese')
print(f'Translation: {response}')
"""
Translation: 工作了一天，我喜欢吃一顿美味的晚餐，看我最喜欢的电视剧，这样做有助于我放松，补充能量。
"""
``` |