LoneStriker commited on
Commit
007d6e9
1 Parent(s): 62d6280

Upload README.md

Browse files
Files changed (1) hide show
  1. README.md +285 -52
README.md CHANGED
@@ -7,16 +7,21 @@ widget:
7
  text: hi
8
  output:
9
  text: ' Hello! How can I assist you today?'
10
-
11
  pipeline_tag: text-generation
12
  ---
 
 
 
 
 
13
 
14
- # 🐗SUS-Chat: Instruction tuning done right
15
 
16
  <div align="center">
17
 
18
  <p align="center">
19
- <img width="200px" src="https://github.com/SUSTech-IDEA/SUS-Chat/raw/main/assets/sustech.svg?sanitize=true">
 
20
  </p>
21
 
22
  <div style="display: inline-block;">
@@ -38,7 +43,7 @@ pipeline_tag: text-generation
38
  <div style="display: inline-block;">
39
 
40
  <a rel="noopener nofollow" href="https://www.modelscope.cn/organization/sustc/">
41
- <img src="https://img.shields.io/badge/ModelScope-sustec-blue" style="margin: 0 0;">
42
  </a>
43
 
44
  </div>
@@ -53,7 +58,7 @@ pipeline_tag: text-generation
53
 
54
  <div style="display: inline-block;">
55
 
56
- <a rel="noopener nofollow" href="https://github.com/SUSTech-IDEA/SUS-Chat/blob/main/MODEL_LICENSE_AGREEMENT.txt">
57
  <img src="https://img.shields.io/badge/Model_License-Model_Agreement-lightblue" style="margin: 0 0;">
58
  </a>
59
 
@@ -69,54 +74,263 @@ pipeline_tag: text-generation
69
 
70
  </div>
71
 
72
- # Inrtoduction
73
 
74
- <img src="https://hackmd.io/_uploads/S1dXCTIHp.png" id="fig-sus"
75
- alt="Figure 1: DALL·E 2023-12-01 11.03.28 - An imposing, majestic wild boar combined with elements of a futuristic transformer robot. The boar itself should be intricately blended with these tra" />
 
 
 
76
 
77
- **SUS-Chat**
78
- 是一个34B的中英文对话模型,由南方科技大学和粤港澳大湾区数字经济研究院联合发布。SUS-Chat-34B模型在数百万高质、多语言的指令数据上进行了微调,在保持基础模型强大的语言能力的同时,SUS-Chat-34B模型通过高质量指令微调改善了模型对人类指令的响应方式并擅长通过思维链的方式模仿人类思考过程。
 
79
 
80
- 它在几乎所有基准测试中超过了所有同尺寸的模型,而且能够更好地满足了复杂多语言任务的实际需求,相比于更大的模型,SUS-Chat-34B仍具有相当竞争力,在我们的综合评测中取得了最先进的表现。
 
81
 
82
- SUS-Chat有力地证明了通过正确的指令微调,学术机构可以在不增加模型参数的情况下,通过开源的数据集和模型,获得更好的性能,
83
- 这弥合了学术界和工业界的在大语言模型上的差距,为学术界和工业界的合作提供了新的可能性。
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
84
 
85
  # Performance
86
 
87
- 为了更好地评估SUS-Chat-34B模型的性能,我们在多个基准测试中进行了评估,并开源了评估框架[TLEM](https://huggingface.co/spaces/SUSTech/tlem),以便于其他研究人员进行复现和比较。
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
88
 
89
- 在TLEM中,我们使用了多个基准测试,包括:MMLU, CMMLU, C-Eval, BBH,
90
- GSM-8K, MATH,
91
- 专注于衡量模型的知识和思维能力,在这些指标中SUS-Chat-34B模型取得了最先进的表现,我们还额外引入了[lm-eval](https://github.com/EleutherAI/lm-evaluation-harness)测试了SUS-Chat和同类模型在winogrande,
92
- hellaswag, arc, truthful-qa的表现, 衡量模型的常识性推理能力和幻觉。
93
 
94
- 综合上看,SUS-Chat-34B模型显著领先于同规模的模型,并取得了最先进的综合性能。
95
 
96
- | model | mmlu-chat | cmmlu-chat | ceval-chat | gsm8k | BBH | MATH | winogrande | arc | hellaswag | truthfulqa | average |
97
- |:------------------|----------:|-----------:|-----------:|------:|------:|------:|-----------:|------:|----------:|-----------:|--------:|
98
- | GPT-4 | 83 | 71 | 69.9 | 91.4 | 86.7 | 45.8 | 87.5 | 94.5 | 91.4 | nan | 80.1333 |
99
- | SUS-Chat-34B | 77.35 | 78.68 | 82.42 | 80.06 | 67.62 | 28.8 | 81.22 | 81.54 | 83.79 | 57.47 | 71.895 |
100
- | Qwen-72B-Chat | 74.52 | 77.02 | 77.22 | 76.57 | 72.63 | 35.9 | 80.58 | 81.29 | 87.02 | 50.64 | 71.339 |
101
- | DeepSeek-67B-Chat | 69.43 | 48.51 | 59.7 | 74.45 | 69.73 | 29.56 | 76.09 | 82.1 | 86.06 | 56.37 | 65.2 |
102
- | OrionStar-34B | 68.51 | 66.88 | 65.13 | 54.36 | 62.88 | 12.8 | 77.27 | 80.19 | 84.54 | 53.24 | 62.58 |
103
- | Yi-34B-Chat | 66.96 | 55.16 | 77.16 | 63.76 | 61.54 | 10.02 | 76.64 | 70.66 | 82.29 | 54.57 | 61.876 |
104
 
105
- <img src="assets/radar.png" id="fig-bench" alt="Figure 2: Benchmark" />
106
 
107
- # 用法
 
 
 
 
 
 
 
108
 
109
- SUS-Chat-34B是标准的LLaMA模型,使用方法和开发环境与大多数其它开源模型相同,可以通过以下方式进行多轮对话
110
 
111
- ``` python
112
- from transformers import AutoModelForCausalLM, AutoTokenizer
 
 
 
 
 
113
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
114
 
115
  def chat_template(messages):
116
  history = ""
117
  for message in messages:
118
  match message:
119
- case {"role": "human", "content": message}:
120
  history += f"### Human: {message}\n\n### Assistant: "
121
  case {"role": "assistant", "content": message}:
122
  history += message
@@ -132,10 +346,12 @@ model = AutoModelForCausalLM.from_pretrained(
132
 
133
  messages = [{"role": "user", "content": "hi"}]
134
 
135
- input_ids = tokenizer.encode(chat_template(messages), return_tensors="pt").to("cuda")
136
- output_ids = model.generate(input_ids.to("cuda"))
 
 
137
  response = tokenizer.decode(
138
- output_ids[0][input_ids.shape[1] :], skip_special_tokens=True
139
  )
140
 
141
  messages.append({"role": "assistant", "content": response})
@@ -144,25 +360,42 @@ messages.append({"role": "assistant", "content": response})
144
 
145
  messages.append({"role": "user", "content": "What is the capital of China?"})
146
 
147
- input_ids = tokenizer.encode(chat_template(messages), return_tensors="pt").to("cuda")
148
- output_ids = model.generate(input_ids.to("cuda"))
 
 
149
  response = tokenizer.decode(
150
- output_ids[0][input_ids.shape[1] :], skip_special_tokens=True
151
  )
152
 
153
  messages.append({"role": "assistant", "content": response})
154
  ```
155
 
156
- # 限制
157
-
158
- SUS-Chat只进行了监督微调,尚未进行人类偏好学习,因此在一些情况下可能会产生不合理的回复,并放大某些语言模型现有的问题,
159
- 包括幻觉、非确定性和累积误差,
160
- 为了实现更有利于下游任务的性能,我们建议相应地调整生成是配置参数。
161
-
162
- # 免责声明
163
-
164
- 我们在训练过程中使用数据合规检查算法,尽力确保训练模型的合规性。由于数据复杂且语言模型使用场景多样,我们无法保证模型在所有情况下生成正确和合理的输出。请注意,模型仍然存在产生问题输出的风险。对于因滥用、误导、非法使用和相关错误信息以及相关数据安全问题而导致的任何风险和问题,我们将不承担责任。
165
-
166
- # 许可
167
-
168
- 该模型完全开发供学术研究和免费商业使用,但需要遵守来自零一万物的[许可](https://github.com/SUSTech-IDEA/SUS-Chat/blob/main/MODEL_LICENSE_AGREEMENT.txt)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
7
  text: hi
8
  output:
9
  text: ' Hello! How can I assist you today?'
 
10
  pipeline_tag: text-generation
11
  ---
12
+ # 🐷SUS-Chat: Instruction tuning done right
13
+
14
+ <p align="left">
15
+ <a href="README_CN.md">中文</a>&nbsp | &nbspEnglish&nbsp
16
+ </p>
17
 
18
+ <br><br>
19
 
20
  <div align="center">
21
 
22
  <p align="center">
23
+ <img src="https://github.com/SUSTech-IDEA/SUS-Chat/raw/main/assets/sustech.svg?sanitize=true" width="200px">
24
+ <img src="https://github.com/SUSTech-IDEA/SUS-Chat/raw/main/assets/ccnl.png?sanitize=true" width="200px">
25
  </p>
26
 
27
  <div style="display: inline-block;">
 
43
  <div style="display: inline-block;">
44
 
45
  <a rel="noopener nofollow" href="https://www.modelscope.cn/organization/sustc/">
46
+ <img src="https://img.shields.io/badge/🤖ModelScope-sustc-blue" style="margin: 0 0;">
47
  </a>
48
 
49
  </div>
 
58
 
59
  <div style="display: inline-block;">
60
 
61
+ <a rel="noopener nofollow" href="https://github.com/01-ai/Yi/blob/main/MODEL_LICENSE_AGREEMENT.txt">
62
  <img src="https://img.shields.io/badge/Model_License-Model_Agreement-lightblue" style="margin: 0 0;">
63
  </a>
64
 
 
74
 
75
  </div>
76
 
77
+ # News
78
 
79
+ - 2023-12-06: Try [SUS-Chat-34B
80
+ chat-ui](https://huggingface.co/spaces/SUSTech/SUS-Chat-34B).
81
+
82
+ - 2023-12-05: SUS-Chat-34B is now available on
83
+ [ModelScope🤖](https://www.modelscope.cn/models/SUSTC/SUS-Chat-34B/summary)
84
 
85
+ - 2023-12-05: SUS-Chat-34B is ranked 2nd in [Open LLM
86
+ leaderboard](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)
87
+ and surpassed all models under 70B.
88
 
89
+ - 2023-12-01: SUS-Chat-34B is now available on
90
+ [HuggingFace🤗](https://huggingface.co/SUSTech/SUS-Chat-34B).
91
 
92
+ # Introduction
93
+
94
+ <img src="https://hackmd.io/_uploads/HJlDtzhBa.png" id="fig-sus"
95
+ alt="Figure 1: DALL·E 2023-12-01 11.03.28 - An imposing, majestic wild boar combined with elements of a futuristic transformer robot. The boar itself should be intricately blended with these tra" />
96
+
97
+ **SUS-Chat-34B** is a 34B bilingual Chinese-English dialogue model,
98
+ jointly released by the **[Southern University of Science and
99
+ Technology](https://huggingface.co/SUSTech)** and
100
+ **[IDEA-CCNL](https://huggingface.co/IDEA-CCNL)**. This model is based
101
+ on [`01-ai/Yi-34B`](https://huggingface.co/01-ai/Yi-34B) and has been
102
+ fine-tuned on millions of high-quality, multilingual instruction data.
103
+ While maintaining the strong language capabilities of the base model,
104
+ the SUS-Chat-34B model has improved the model’s response to human
105
+ instructions through high-quality instruction fine-tuning and excels at
106
+ imitating human thought processes through chains of thought. It
107
+ introduces inter-instruction attention sharing in long texts, expanding
108
+ the window size from 4K to 8K, significantly enhancing the usability of
109
+ multi-turn dialogues.
110
+
111
+ It has surpassed all models of the same size in almost all benchmark
112
+ tests and is better suited to meet the practical needs of complex
113
+ multilingual tasks. Compared to larger models, SUS-Chat-34B remains
114
+ highly competitive and has achieved state-of-the-art performance in our
115
+ comprehensive evaluations.
116
+
117
+ SUS-Chat-34B model has the following highlights:
118
+
119
+ 1. Large-scale complex instruction following data: Trained with 1.4
120
+ billion tokens of high-quality complex instruction data, covering
121
+ Chinese and English, multi-turn dialogues, mathematics, reasoning,
122
+ and various other types of instruction data;
123
+ 2. Strong performance in general tasks: The SUS-Chat-34B model excels
124
+ in numerous mainstream Chinese and English tasks, surpassing other
125
+ open-source instruction fine-tuned models of the same parameter
126
+ scale. It also competes well against models with larger parameter
127
+ scales;
128
+ 3. Longer context window and excellent multi-turn dialogue
129
+ capabilities: Currently, SUS-Chat-34B supports an 8K context window,
130
+ and is trained with a large amount of multi-turn instruction and
131
+ single-multi-turn mixed data, demonstrating remarkable capabilities
132
+ in long-text dialogue information focus and instruction follow-up.
133
+
134
+ SUS-Chat powerfully demonstrates that through the right instruction
135
+ fine-tuning, academic institutions can achieve better performance
136
+ without increasing model parameters, using open-source datasets and
137
+ models. This bridges the gap between academia and industry in large
138
+ language models and opens new possibilities for collaboration between
139
+ academic and industrial sectors.
140
 
141
  # Performance
142
 
143
+ To better evaluate the performance of the SUS-Chat-34B model, we
144
+ conducted assessments across multiple benchmark tests and have
145
+ open-sourced the evaluation framework
146
+ [TLEM](https://huggingface.co/spaces/SUSTech/tlem) to facilitate
147
+ replication and comparison by other researchers.
148
+
149
+ In TLEM, we utilized various benchmark tests including MMLU, CMMLU,
150
+ C-Eval, BBH, GSM-8K, and MATH, to measure the model’s knowledge and
151
+ thinking capabilities. In these metrics, the SUS-Chat-34B model achieved
152
+ state-of-the-art performance. Additionally, we incorporated
153
+ [lm-eval](https://github.com/EleutherAI/lm-evaluation-harness) to test
154
+ SUS-Chat and similar models on winogrande, hellaswag, arc, and
155
+ truthful-qa, assessing the model’s common-sense reasoning ability and
156
+ susceptibility to illusions.
157
+
158
+ Overall, the SUS-Chat-34B model significantly outperformed models of
159
+ similar scale and achieved the most advanced comprehensive performance.
160
+
161
+ <img
162
+ src="https://github.com/SUSTech-IDEA/SUS-Chat/raw/main/assets/radar.png"
163
+ id="fig-bench" alt="Figure 2: Benchmark" />
164
+
165
+ <div>
166
+
167
+ <table>
168
+ <colgroup>
169
+ <col style="width: 50%" />
170
+ <col style="width: 50%" />
171
+ </colgroup>
172
+ <tbody>
173
+ <tr class="odd">
174
+ <td style="text-align: center;"><div width="50.0%"
175
+ data-layout-align="center">
176
+ <h2 id="english-understanding">English Understanding</h2>
177
+ <table>
178
+ <thead>
179
+ <tr class="header">
180
+ <th style="text-align: right;">Model</th>
181
+ <th style="text-align: center;">mmlu (0-shot)</th>
182
+ </tr>
183
+ </thead>
184
+ <tbody>
185
+ <tr class="odd">
186
+ <td style="text-align: right;">GPT-4</td>
187
+ <td style="text-align: center;">83</td>
188
+ </tr>
189
+ <tr class="even">
190
+ <td style="text-align: right;">SUS-Chat-34B</td>
191
+ <td style="text-align: center;"><u>74.35</u></td>
192
+ </tr>
193
+ <tr class="odd">
194
+ <td style="text-align: right;">Qwen-72b-Chat</td>
195
+ <td style="text-align: center;"><strong>74.52</strong></td>
196
+ </tr>
197
+ <tr class="even">
198
+ <td style="text-align: right;">Deepseek-68b-Chat</td>
199
+ <td style="text-align: center;">69.43</td>
200
+ </tr>
201
+ <tr class="odd">
202
+ <td style="text-align: right;">OrionStar-Yi-34B-Chat</td>
203
+ <td style="text-align: center;">68.51</td>
204
+ </tr>
205
+ <tr class="even">
206
+ <td style="text-align: right;">Yi-34B-Chat</td>
207
+ <td style="text-align: center;">66.96</td>
208
+ </tr>
209
+ </tbody>
210
+ </table>
211
+ </div></td>
212
+ <td style="text-align: center;"><div width="50.0%"
213
+ data-layout-align="center">
214
+ <h2 id="chinese-capabilities">Chinese Capabilities</h2>
215
+ <table>
216
+ <colgroup>
217
+ <col style="width: 34%" />
218
+ <col style="width: 32%" />
219
+ <col style="width: 32%" />
220
+ </colgroup>
221
+ <thead>
222
+ <tr class="header">
223
+ <th style="text-align: right;">Model</th>
224
+ <th style="text-align: center;">cmmlu (0-shot)</th>
225
+ <th style="text-align: center;">C-Eval (0-shot)<a href="#fn1"
226
+ class="footnote-ref" id="fnref1"
227
+ role="doc-noteref"><sup>1</sup></a></th>
228
+ </tr>
229
+ </thead>
230
+ <tbody>
231
+ <tr class="odd">
232
+ <td style="text-align: right;">GPT-4</td>
233
+ <td style="text-align: center;">71</td>
234
+ <td style="text-align: center;">69.9</td>
235
+ </tr>
236
+ <tr class="even">
237
+ <td style="text-align: right;">SUS-Chat-34B</td>
238
+ <td style="text-align: center;"><strong>78.68</strong></td>
239
+ <td style="text-align: center;"><strong>82.42</strong></td>
240
+ </tr>
241
+ <tr class="odd">
242
+ <td style="text-align: right;">Qwen-72b-Chat</td>
243
+ <td style="text-align: center;"><u>77.02</u></td>
244
+ <td style="text-align: center;"><u>77.22</u></td>
245
+ </tr>
246
+ <tr class="even">
247
+ <td style="text-align: right;">Deepseek-68b-Chat</td>
248
+ <td style="text-align: center;">48.51</td>
249
+ <td style="text-align: center;">59.7</td>
250
+ </tr>
251
+ <tr class="odd">
252
+ <td style="text-align: right;">OrionStar-Yi-34B-Chat</td>
253
+ <td style="text-align: center;">66.88</td>
254
+ <td style="text-align: center;">65.13</td>
255
+ </tr>
256
+ <tr class="even">
257
+ <td style="text-align: right;">Yi-34B-Chat</td>
258
+ <td style="text-align: center;">55.16</td>
259
+ <td style="text-align: center;">77.16</td>
260
+ </tr>
261
+ </tbody>
262
+ </table>
263
+ </div></td>
264
+ </tr>
265
+ </tbody>
266
+ </table>
267
+ <section id="footnotes" class="footnotes footnotes-end-of-document"
268
+ role="doc-endnotes">
269
+ <hr />
270
+ <ol>
271
+ <li id="fn1"><p>C-Eval results are evaluated on the validation
272
+ datasets<a href="#fnref1" class="footnote-back"
273
+ role="doc-backlink">↩︎</a></p></li>
274
+ </ol>
275
+ </section>
276
 
277
+ </div>
 
 
 
278
 
279
+ ## Math & Reasoning
280
 
281
+ | Model | gsm8k (0-shot) | MATH (0-shot) | BBH (0-shot) |
282
+ |----------------------:|:--------------:|:-------------:|:------------:|
283
+ | GPT-4 | 91.4 | 45.8 | 86.7 |
284
+ | SUS-Chat-34B | **80.06** | 28.7 | 67.62 |
285
+ | Qwen-72b-Chat | <u>76.57</u> | **35.9** | **72.63** |
286
+ | Deepseek-68b-Chat | 74.45 | <u>29.56</u> | <u>69.73</u> |
287
+ | OrionStar-Yi-34B-Chat | 54.36 | 12.8 | 62.88 |
288
+ | Yi-34B-Chat | 63.76 | 10.02 | 61.54 |
289
 
290
+ ## More Tasks
291
 
292
+ | Model | winogrande (5-shot) | arc (25-shot) | hellaswag (10-shot) | TruthfulQA mc1 (0-shot) | TruthfulQA mc2 (0-shot) |
293
+ |----------------------:|:-------------------:|:-------------:|:-------------------:|:-----------------------:|:-----------------------:|
294
+ | GPT-4 | — | 94.5 | 91.4 | 59.00 | — |
295
+ | SUS-Chat-34B | **81.22** | <u>81.54</u> | 83.79 | **40.64** | **57.47** |
296
+ | Qwen-72b-Chat | 76.09 | **82.10** | <u>86.06</u> | 39.17 | <u>56.37</u> |
297
+ | Deepseek-68b-Chat | <u>80.58</u> | 81.29 | **87.02** | <u>40.02</u> | 50.64 |
298
+ | OrionStar-Yi-34B-Chat | 77.27 | 80.19 | 84.54 | 36.47 | 53.24 |
299
+ | Yi-34B-Chat | 76.64 | 70.66 | 82.29 | 38.19 | 54.57 |
300
 
301
+ ## Overall
302
 
303
+ | Model | Average |
304
+ |----------------------:|:---------:|
305
+ | SUS-Chat-34B | **69.05** |
306
+ | Qwen-72b-Chat | 68.41 |
307
+ | Deepseek-68b-Chat | 62.91 |
308
+ | OrionStar-Yi-34B-Chat | 60.21 |
309
+ | Yi-34B-Chat | 59.72 |
310
 
311
+ To reproduce the results, please start a corresponding vllm server and
312
+ refer to
313
+ [here](https://sustech-tlem.static.hf.space/index.html#start-evaluating-your-model-in-3-line).
314
+
315
+ # Usage
316
+
317
+ SUS-Chat-34B is a standard LLaMA model and should be seamlessly
318
+ compatible with the LLaMA ecosystem. We provide the following example to
319
+ demonstrate how it can be used for multi-turn dialogues.
320
+
321
+ Feel free to [open an
322
+ issue](https://github.com/SUSTech-IDEA/SUS-Chat/issues) if you have any
323
+ questions.
324
+
325
+ ``` python
326
+ from transformers import AutoModelForCausalLM, AutoTokenizer # 🤗 Transformers, or
327
+ # from modelscope import AutoModelForCausalLM, AutoTokenizer # 🤖 ModelScope
328
 
329
  def chat_template(messages):
330
  history = ""
331
  for message in messages:
332
  match message:
333
+ case {"role": "user", "content": message}:
334
  history += f"### Human: {message}\n\n### Assistant: "
335
  case {"role": "assistant", "content": message}:
336
  history += message
 
346
 
347
  messages = [{"role": "user", "content": "hi"}]
348
 
349
+ input_ids = tokenizer.encode(
350
+ chat_template(messages), return_tensors="pt", add_special_tokens=False
351
+ ).to("cuda")
352
+ output_ids = model.generate(input_ids.to("cuda"), max_length=256)
353
  response = tokenizer.decode(
354
+ output_ids[0][input_ids.shape[1] :], skip_special_tokens=False
355
  )
356
 
357
  messages.append({"role": "assistant", "content": response})
 
360
 
361
  messages.append({"role": "user", "content": "What is the capital of China?"})
362
 
363
+ input_ids = tokenizer.encode(
364
+ chat_template(messages), return_tensors="pt", add_special_tokens=False
365
+ ).to("cuda")
366
+ output_ids = model.generate(input_ids.to("cuda"), max_length=256)
367
  response = tokenizer.decode(
368
+ output_ids[0][input_ids.shape[1] :], skip_special_tokens=False
369
  )
370
 
371
  messages.append({"role": "assistant", "content": response})
372
  ```
373
 
374
+ # Limitations
375
+
376
+ SUS-Chat has only undergone supervised fine-tuning and has not yet been
377
+ trained on human preference learning. As a result, it may produce
378
+ unreasonable responses in some situations and exacerbate existing issues
379
+ in language models, including hallucinations, non-determinism, and
380
+ cumulative errors. To achieve better performance for downstream tasks,
381
+ we recommend adjusting the generation configuration parameters
382
+ accordingly.
383
+
384
+ # Disclaimer
385
+
386
+ During the training process, we used data compliance check algorithms to
387
+ ensure the compliance of the training model as much as possible. Due to
388
+ the complexity of the data and the diverse use cases of language models,
389
+ we cannot guarantee that the model will produce correct and reasonable
390
+ outputs in all scenarios. Please be aware that there is still a risk of
391
+ the model generating problematic outputs. We will not be responsible for
392
+ any risks or issues arising from misuse, misguidance, illegal use, and
393
+ related misinformation, as well as data security issues related to the
394
+ model.
395
+
396
+ # License
397
+
398
+ This model is developed entirely for academic research and free
399
+ commercial use, but it must adhere to the
400
+ [license](https://github.com/01-ai/Yi/blob/main/MODEL_LICENSE_AGREEMENT.txt)
401
+ from [01-ai](https://huggingface.co/01-ai).