sail
/

Text Generation
Transformers
Safetensors
qwen2
multilingual
sea
sailor
sft
chat
instruction
conversational
text-generation-inference
dreamerdeo commited on
Commit
20fd49c
1 Parent(s): 5831e6e

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +27 -10
README.md CHANGED
@@ -51,7 +51,7 @@ The pre-training corpus heavily leverages the publicly available corpus, includi
51
  [SlimPajama](https://huggingface.co/datasets/cerebras/SlimPajama-627B),
52
  [SkyPile](https://huggingface.co/datasets/Skywork/SkyPile-150B),
53
  [CC100](https://huggingface.co/datasets/cc100) and [MADLAD-400](https://huggingface.co/datasets/allenai/MADLAD-400).
54
- The instruction tuning corpus are all public available including
55
  [aya_collection](https://huggingface.co/datasets/CohereForAI/aya_collection),
56
  [aya_dataset](https://huggingface.co/datasets/CohereForAI/aya_dataset),
57
  [OpenOrca](https://huggingface.co/datasets/Open-Orca/OpenOrca).
@@ -70,25 +70,42 @@ Here provides a code snippet to show you how to load the tokenizer and model and
70
 
71
  ```python
72
  from transformers import AutoModelForCausalLM, AutoTokenizer
73
- device = "cuda" # the device to load the model
74
 
75
- model = AutoModelForCausalLM.from_pretrained("sail/Sailor-7B", device_map="auto")
76
- tokenizer = AutoTokenizer.from_pretrained("sail/Sailor-7B")
 
 
 
 
 
 
 
 
 
 
77
 
78
- input_message = "Model bahasa adalah model probabilistik"
79
- ### The given Indonesian input translates to 'A language model is a probabilistic model of.'
 
 
 
 
 
 
 
80
 
81
- model_inputs = tokenizer([input_message], return_tensors="pt").to(device)
 
82
 
83
  generated_ids = model.generate(
84
- model_inputs.input_ids,
85
- max_new_tokens=64
86
  )
87
 
88
  generated_ids = [
89
  output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
90
  ]
91
-
92
  response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
93
  print(response)
94
  ```
 
51
  [SlimPajama](https://huggingface.co/datasets/cerebras/SlimPajama-627B),
52
  [SkyPile](https://huggingface.co/datasets/Skywork/SkyPile-150B),
53
  [CC100](https://huggingface.co/datasets/cc100) and [MADLAD-400](https://huggingface.co/datasets/allenai/MADLAD-400).
54
+ The instruction tuning corpus are all publicly available including
55
  [aya_collection](https://huggingface.co/datasets/CohereForAI/aya_collection),
56
  [aya_dataset](https://huggingface.co/datasets/CohereForAI/aya_dataset),
57
  [OpenOrca](https://huggingface.co/datasets/Open-Orca/OpenOrca).
 
70
 
71
  ```python
72
  from transformers import AutoModelForCausalLM, AutoTokenizer
73
+ device = "cuda"
74
 
75
+ model = AutoModelForCausalLM.from_pretrained(
76
+ 'sail/Sailor-1.8B-Chat',
77
+ torch_dtype="auto",
78
+ device_map="auto"
79
+ )
80
+
81
+ tokenizer = AutoTokenizer.from_pretrained('sail/Sailor-1.8B-Chat')
82
+ system_prompt= 'You are a helpful assistant'
83
+
84
+ prompt = "Beri saya pengenalan singkat tentang model bahasa besar."
85
+ # prompt = "Hãy cho tôi một giới thiệu ngắn gọn về mô hình ngôn ngữ lớn."
86
+ # prompt = "ให้ฉันแนะนำสั้น ๆ เกี่ยวกับโมเดลภาษาขนาดใหญ่"
87
 
88
+ messages = [
89
+ {"role": "system", "content": system_prompt},
90
+ {"role": "question", "content": prompt}
91
+ ]
92
+ text = tokenizer.apply_chat_template(
93
+ messages,
94
+ tokenize=False,
95
+ add_generation_prompt=True
96
+ )
97
 
98
+ model_inputs = tokenizer([text], return_tensors="pt").to(device)
99
+ input_ids = model_inputs.input_ids.to(device)
100
 
101
  generated_ids = model.generate(
102
+ input_ids,
103
+ max_new_tokens=512,
104
  )
105
 
106
  generated_ids = [
107
  output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
108
  ]
 
109
  response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
110
  print(response)
111
  ```