yi-01-ai commited on
Commit
a068e6b
1 Parent(s): 54fb89a

Auto Sync from git://github.com/01-ai/Yi.git/commit/e405fcdc8651a5763da53034d08d6783dd3eeffd

Browse files
Files changed (1) hide show
  1. README.md +184 -72
README.md CHANGED
@@ -26,63 +26,101 @@ pipeline_tag: text-generation
26
  </br>
27
 
28
  <div style="display: inline-block;">
29
- <a rel="noopener nofollow" href="https://github.com/01-ai/Yi/actions/workflows/build_docker_image.yml">
30
- <img src="https://github.com/01-ai/Yi/actions/workflows/build_docker_image.yml/badge.svg" style="margin: 0 0;">
31
  </a>
32
  </div>
33
 
34
  <div style="display: inline-block;">
35
- <a href="https://huggingface.co/01-ai">
36
- <img src="https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-01--ai-blue" style="margin: 0 0;">
37
  </a>
38
  </div>
39
 
40
  <div style="display: inline-block;">
41
- <a rel="noopener nofollow" href="https://www.modelscope.cn/organization/01ai/">
42
- <img src="https://img.shields.io/badge/ModelScope-01--ai-blue" style="margin: 0 0;">
43
  </a>
44
  </div>
45
 
46
  <div style="display: inline-block;">
47
- <a rel="noopener nofollow" href="https://wisemodel.cn/organization/01.AI">
48
- <img src="https://img.shields.io/badge/WiseModel-01--ai-blue" style="margin: 0 0;">
49
  </a>
50
  </div>
51
 
52
- <div style="display: inline-block;">
53
- <a rel="noopener nofollow" href="https://replicate.com/01-ai">
54
- <img src="https://img.shields.io/badge/Replicate-01--ai-blue?logo=data:image/svg%2bxml;base64,PHN2ZyB2ZXJzaW9uPSIxLjEiIHhtbG5zPSJodHRwOi8vd3d3LnczLm9yZy8yMDAwL3N2ZyIgeG1sbnM6eGxpbms9Imh0dHA6Ly93d3cudzMub3JnLzE5OTkveGxpbmsiIHg9IjBweCIgeT0iMHB4IiB2aWV3Qm94PSIwIDAgMTAwMCAxMDAwIiBjbGFzcz0ibG9nbyIgZmlsbD0iY3VycmVudENvbG9yIiB4bWw6c3BhY2U9InByZXNlcnZlIj4KICA8Zz4KICAgIDxwb2x5Z29uIHBvaW50cz0iMTAwMCw0MjcuNiAxMDAwLDU0MC42IDYwMy40LDU0MC42IDYwMy40LDEwMDAgNDc3LDEwMDAgNDc3LDQyNy42IAkiPjwvcG9seWdvbj4KICAgIDxwb2x5Z29uIHBvaW50cz0iMTAwMCwyMTMuOCAxMDAwLDMyNyAzNjQuOCwzMjcgMzY0LjgsMTAwMCAyMzguNCwxMDAwIDIzOC40LDIxMy44IAkiPjwvcG9seWdvbj4KICAgIDxwb2x5Z29uIHBvaW50cz0iMTAwMCwwIDEwMDAsMTEzLjIgMTI2LjQsMTEzLjIgMTI2LjQsMTAwMCAwLDEwMDAgMCwwIAkiPjwvcG9seWdvbj4KICA8L2c+Cjwvc3ZnPg==" style="margin: 0 0;">
55
- </a>
56
  </div>
57
 
58
- <div style="display: inline-block;">
59
- <a rel="noopener nofollow" href="https://github.com/01-ai/Yi/blob/main/LICENSE">
60
- <img src="https://img.shields.io/badge/Code_License-Apache_2.0-lightblue" style="margin: 0 0;">
61
- </a>
62
- </div>
63
 
64
- <div style="display: inline-block;">
65
- <a rel="noopener nofollow" href="https://github.com/01-ai/Yi/blob/main/MODEL_LICENSE_AGREEMENT.txt">
66
- <img src="https://img.shields.io/badge/Model_License-Model_Agreement-lightblue" style="margin: 0 0;">
67
- </a>
68
- </div>
69
 
70
- <div style="display: inline-block;">
71
- <a rel="noopener nofollow" href="mailto:oss@01.ai">
72
- <img src="https://img.shields.io/badge/✉️-yi@01.ai-FFE01B" style="margin: 0 0;">
73
- </a>
74
- </div>
75
 
76
- </div>
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
77
 
78
- ## Introduction
79
 
80
- The **Yi** series models are large language models trained from scratch by
81
- developers at [01.AI](https://01.ai/).
82
 
83
- ## News
84
 
85
- <details open>
 
 
 
 
 
 
 
 
 
 
 
 
86
  <summary>🎯 <b>2023/11/23</b>: The chat models are open to public.</summary>
87
 
88
  This release contains two chat models based on previous released base models, two 8-bits models quantized by GPTQ, two 4-bits models quantized by AWQ.
@@ -100,11 +138,11 @@ You can try some of them interactively at:
100
  - [Replicate](https://replicate.com/01-ai)
101
  </details>
102
 
103
- <details open>
104
  <summary>🔔 <b>2023/11/23</b>: The Yi Series Models Community License Agreement is updated to v2.1.</summary>
105
  </details>
106
 
107
- <details>
108
  <summary>🔥 <b>2023/11/08</b>: Invited test of Yi-34B chat model.</summary>
109
 
110
  Application form:
@@ -131,28 +169,91 @@ sequence length and can be extended to 32K during inference time.
131
 
132
  </details>
133
 
134
- ## Ecosystem
 
 
 
 
 
135
 
136
- 🤗 You are encouraged to create a PR and share your awesome work built on top of
137
- the Yi series models.
138
 
139
- - Serving
140
- - [ScaleLLM](https://github.com/vectorch-ai/ScaleLLM#supported-models): Efficiently run Yi models locally.
141
- - Quantization
142
- - [TheBloke/Yi-34B-GGUF](https://huggingface.co/TheBloke/Yi-34B-GGUF)
143
- - [TheBloke/Yi-34B-GPTQ](https://huggingface.co/TheBloke/Yi-34B-GPTQ)
144
- - Finetuning
145
- - [NousResearch/Nous-Capybara-34B](https://huggingface.co/NousResearch/Nous-Capybara-34B)
146
- - [SUSTech/SUS-Chat-34B](https://huggingface.co/SUSTech/SUS-Chat-34B): This
147
- model ranks first among all models below 70B and has outperformed the twice
148
- larger
149
- [deepseek-llm-67b-chat](https://huggingface.co/deepseek-ai/deepseek-llm-67b-chat).
150
- You can check the result in [🤗 Open LLM
151
- Leaderboard](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard).
152
 
153
- ## Model Performance
154
 
155
- ### Base Model Performance
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
156
 
157
  | Model | MMLU | CMMLU | C-Eval | GAOKAO | BBH | Common-sense Reasoning | Reading Comprehension | Math & Code |
158
  | :------------ | :------: | :------: | :------: | :------: | :------: | :--------------------: | :-------------------: | :---------: |
@@ -193,7 +294,7 @@ is derived by averaging the scores on the remaining tasks. Since the scores for
193
  these two tasks are generally lower than the average, we believe that
194
  Falcon-180B's performance was not underestimated.
195
 
196
- ### Chat Model Performance
197
 
198
  | Model | MMLU | MMLU | CMMLU | CMMLU | C-Eval(val)<sup>*</sup> | C-Eval(val)<sup>*</sup> | Truthful QA | BBH | BBH | GSM8k | GSM8k |
199
  | ----------------------- | --------- | --------- | --------- | --------- | ----------------------- | ----------------------- | ----------- | --------- | --------- | --------- | --------- |
@@ -215,7 +316,7 @@ We evaluated various benchmarks using both zero-shot and few-shot methods, excep
215
 
216
  <strong>*</strong>: C-Eval results are evaluated on the validation datasets
217
 
218
- ### Quantized Chat Model Performance
219
 
220
  We also provide both 4-bit (AWQ) and 8-bit (GPTQ) quantized Yi chat models. Evaluation results on various benchmarks have shown that the quantized models have negligible losses. Additionally, they reduce the memory footprint size. After testing different configurations of prompts and generation lengths, we highly recommend following the guidelines in the memory footprint table below when selecting a device to run our models.
221
 
@@ -230,7 +331,7 @@ We also provide both 4-bit (AWQ) and 8-bit (GPTQ) quantized Yi chat models. Eval
230
 
231
  Note: All the numbers in the table represent the minimum recommended memory for running models of the corresponding size.
232
 
233
- ### Limitations of Chat Model
234
 
235
  The released chat model has undergone exclusive training using Supervised Fine-Tuning (SFT). Compared to other standard chat models, our model produces more diverse responses, making it suitable for various downstream tasks, such as creative scenarios. Furthermore, this diversity is expected to enhance the likelihood of generating higher quality responses, which will be advantageous for subsequent Reinforcement Learning (RL) training.
236
 
@@ -242,12 +343,24 @@ However, this higher diversity might amplify certain existing issues, including:
242
 
243
  To achieve more coherent and consistent responses, it is advisable to adjust generation configuration parameters such as`temperature`,`top_p`, or`top_k`. These adjustments can help in the balance between creativity and coherence in the model's outputs.
244
 
 
 
245
 
 
246
 
247
- ## Usage
248
 
249
- Feel free to [create an issue](https://github.com/01-ai/Yi/issues/new) if you
250
- encounter any problem when using the **Yi** series models.
 
 
 
 
 
 
 
 
 
251
 
252
  ### 1. Prepare development environment
253
 
@@ -377,7 +490,7 @@ The Arctic is a place of great beauty. The ice and snow are a
377
  For more advanced usage, please refer to the
378
  [doc](https://github.com/01-ai/Yi/tree/main/demo).
379
 
380
- #### 3.3 Finetuning from the base model:
381
 
382
  ```bash
383
  bash finetune/scripts/run_sft_Yi_6b.sh
@@ -430,15 +543,11 @@ python quantization/awq/eval_quantized_model.py \
430
 
431
  For more detailed explanation, please read the [doc](https://github.com/01-ai/Yi/tree/main/quantization/awq)
432
 
433
- ## FAQ
434
-
435
- 1. **What dataset was this trained with?**
436
 
437
- The dataset we use contains Chinese & English only. We used approximately 3T
438
- tokens. The detailed number and its construction will be described in the
439
- upcoming technical report.
440
 
441
- ## Disclaimer
442
 
443
  We use data compliance checking algorithms during the training process, to
444
  ensure the compliance of the trained model to the best of our ability. Due to
@@ -449,12 +558,15 @@ problematic outputs. We will not be responsible for any risks and issues
449
  resulting from misuse, misguidance, illegal usage, and related misinformation,
450
  as well as any associated data security concerns.
451
 
452
- ## License
 
 
 
453
 
454
  The source code in this repo is licensed under the [Apache 2.0
455
  license](https://github.com/01-ai/Yi/blob/main/LICENSE). The Yi series models
456
  are fully open for academic research and free commercial usage with permission
457
- via applications. All usage must adhere to the [Model License
458
- Agreement 2.0](https://github.com/01-ai/Yi/blob/main/MODEL_LICENSE_AGREEMENT.txt).
459
- To apply for the official commercial license, please contact us
460
- ([yi@01.ai](mailto:yi@01.ai)).
 
26
  </br>
27
 
28
  <div style="display: inline-block;">
29
+ <a href="https://github.com/01-ai/Yi/actions/workflows/build_docker_image.yml">
30
+ <img src="https://github.com/01-ai/Yi/actions/workflows/build_docker_image.yml/badge.svg">
31
  </a>
32
  </div>
33
 
34
  <div style="display: inline-block;">
35
+ <a href="https://github.com/01-ai/Yi/blob/main/LICENSE">
36
+ <img src="https://img.shields.io/badge/Code_License-Apache_2.0-lightblue">
37
  </a>
38
  </div>
39
 
40
  <div style="display: inline-block;">
41
+ <a href="https://github.com/01-ai/Yi/blob/main/MODEL_LICENSE_AGREEMENT.txt">
42
+ <img src="https://img.shields.io/badge/Model_License-Yi_License-lightblue">
43
  </a>
44
  </div>
45
 
46
  <div style="display: inline-block;">
47
+ <a href="mailto:oss@01.ai">
48
+ <img src="https://img.shields.io/badge/✉️-yi@01.ai-FFE01B">
49
  </a>
50
  </div>
51
 
52
+ <div align="center">
53
+ <h3 align="center">Building the Next Generation of Open-Source and Bilingual LLMs</h3>
 
 
54
  </div>
55
 
56
+ <p align="center">
57
+ 🤗 <a href="https://huggingface.co/01-ai" target="_blank">Hugging Face</a> • 🤖 <a href="https://www.modelscope.cn/organization/01ai/" target="_blank">ModelScope</a> • ✡️ <a href="https://wisemodel.cn/organization/01.AI" target="_blank">WiseModel</a>
58
+ </p>
 
 
59
 
60
+ <p align="center">
61
+ 👋 Join us 💬 <a href="https://github.com/01-ai/Yi/issues/43#issuecomment-1827285245" target="_blank"> WeChat (Chinese) </a>!
62
+ </p>
 
 
63
 
64
+ <!-- DO NOT REMOVE ME -->
 
 
 
 
65
 
66
+ ---
67
+
68
+ <details open>
69
+ <summary></b>📕 Table of Contents</b></summary>
70
+
71
+ - [🟢 What is Yi?](#-what-is-yi)
72
+ - [📌 Introduction](#-introduction)
73
+ - [🎉 News](#-news)
74
+ - [🟢 Why Yi?](#-why-yi)
75
+ - [🌎 Ecosystem](#-ecosystem)
76
+ - [💦 Upstream](#-upstream)
77
+ - [🌊 Downstream](#-downstream)
78
+ - [🔗 Serving](#-serving)
79
+ - [⚙️ Quantitation](#️-quantitation)
80
+ - [🛠️ Fine-tuning](#️-fine-tuning)
81
+ - [📌 Benchmarks](#-benchmarks)
82
+ - [📊 Base model performance](#-base-model-performance)
83
+ - [📊 Chat model performance](#-chat-model-performance)
84
+ - [📊 Quantized chat model performance](#-quantized-chat-model-performance)
85
+ - [⛔️ Limitations of chat model](#️-limitations-of-chat-model)
86
+ - [🟢 Who can use Yi?](#-who-can-use-yi)
87
+ - [🟢 How to use Yi?](#-how-to-use-yi)
88
+ - [1. Prepare development environment](#1-prepare-development-environment)
89
+ - [1.1 Docker](#11-docker)
90
+ - [1.2 Local development environment](#12-local-development-environment)
91
+ - [2. Download the model (optional)](#2-download-the-model-optional)
92
+ - [3. Examples](#3-examples)
93
+ - [3.1 Use the chat model](#31-use-the-chat-model)
94
+ - [3.2 Use the base model](#32-use-the-base-model)
95
+ - [3.3 Finetune from the base model](#33-finetune-from-the-base-model)
96
+ - [3.4 Quantization](#34-quantization)
97
+ - [GPT-Q](#gpt-q)
98
+ - [AWQ](#awq)
99
+ - [🟢 Misc.](#-misc)
100
+ - [📡 Disclaimer](#-disclaimer)
101
+ - [🪪 License](#-license)
102
+
103
+ </details>
104
 
105
+ # 🟢 What is Yi?
106
 
107
+ ## 📌 Introduction
 
108
 
109
+ - 🤖 The Yi series models are the next generation of open source large language models trained from strach by [01.AI](https://01.ai/).
110
 
111
+ - 🙌 Targeted as a bilingual language model and trained on 3T multilingual corpus, the Yi series models become one of the strongest LLM worldwide, showing promise in language understanding, commonsense reasoning, reading comprehension, and more. For example,
112
+
113
+ - For English language capability, the Yi series models ranked 2nd (just behind GPT-4), outperforming other LLMs (such as LLaMA2-chat-70B, Claude 2, and ChatGPT) on the [AlpacaEval Leaderboard](https://tatsu-lab.github.io/alpaca_eval/) in Dec 2023.
114
+
115
+ - For Chinese language capability, the Yi series models landed in 2nd place (following GPT4), surpassing other LLMs (such as Baidu ERNIE, Qwen, and Baichuan) on the [SuperCLUE](https://www.superclueai.com/) in Oct 2023.
116
+
117
+ - 🙏 (Credits to LLaMA) Thanks to the Transformer and LLaMA open-source communities, as they reducing the efforts required to build from scratch and enabling the utilization of the same tools within the AI ecosystem. If you're interested in Yi's adoption of LLaMA architecture and license usage policy, see [Yi's relation with LLaMA](./docs/yi_relation_llama.md).
118
+
119
+ <div align="right"> [ <a href="#building-the-next-generation-of-open-source-and-bilingual-llms">Back to top ⬆️ </a> ] </div>
120
+
121
+ ## 🎉 News
122
+
123
+ <details>
124
  <summary>🎯 <b>2023/11/23</b>: The chat models are open to public.</summary>
125
 
126
  This release contains two chat models based on previous released base models, two 8-bits models quantized by GPTQ, two 4-bits models quantized by AWQ.
 
138
  - [Replicate](https://replicate.com/01-ai)
139
  </details>
140
 
141
+ <details>
142
  <summary>🔔 <b>2023/11/23</b>: The Yi Series Models Community License Agreement is updated to v2.1.</summary>
143
  </details>
144
 
145
+ <details>
146
  <summary>🔥 <b>2023/11/08</b>: Invited test of Yi-34B chat model.</summary>
147
 
148
  Application form:
 
169
 
170
  </details>
171
 
172
+ <div align="right"> [ <a href="#building-the-next-generation-of-open-source-and-bilingual-llms">Back to top ⬆️ </a> ] </div>
173
+
174
+
175
+ # 🟢 Why Yi?
176
+
177
+ ## 🌎 Ecosystem
178
 
179
+ Yi has a comprehensive ecosystem, offering a range of tools, services, and models to enrich your experiences and maximize productivity.
 
180
 
181
+ - [💦 Upstream](#-upstream)
182
+ - [🌊 Downstream](#-downstream)
183
+ - [🔗 Serving](#-serving)
184
+ - [⚙️ Quantitation](#️-quantitation)
185
+ - [🛠️ Fine-tuning](#️-fine-tuning)
 
 
 
 
 
 
 
 
186
 
187
+ ### 💦 Upstream
188
 
189
+ The Yi series models follow the same model architecture as LLaMA. By choosing Yi, you can leverage existing tools, libraries, and resources within the LLaMA ecosystem, eliminating the need to create new tools and enhancing development efficiency.
190
+
191
+ For example, the Yi series models are saved in the format of the LLaMA model. You can directly use `LLaMAForCausalLM` and `LLaMATokenizer` to load the model. For more information, see [Use the chat model](#31-use-the-chat-model).
192
+
193
+ ```python
194
+ from transformers import AutoModelForCausalLM, AutoTokenizer
195
+
196
+ tokenizer = AutoTokenizer.from_pretrained("01-ai/Yi-34b", use_fast=False)
197
+
198
+ model = AutoModelForCausalLM.from_pretrained("01-ai/Yi-34b", device_map="auto")
199
+ ```
200
+
201
+ ### 🌊 Downstream
202
+
203
+ > 💡 Tip
204
+ >
205
+ > - Feel free to create a PR and share the fantastic work you've built using the Yi series models.
206
+ >
207
+ > - To help others quickly understand your work, it is recommended to use the format of `<model-name>: <model-intro> + <model-highlights>`.
208
+
209
+ #### 🔗 Serving
210
+
211
+ If you want to get up with Yi in a few minutes, you can use the following services built upon Yi.
212
+
213
+ - [Yi-34B-Chat](https://platform.lingyiwanwu.com/) (Yi official beta): you can chat with it. **Note** that currently it's available through a whitelist. Welcome to apply and experience it firsthand!
214
+
215
+ - [Yi-6B-Chat (Replicate)](https://replicate.com/01-ai): you can use this model with more options by setting additional parameters and calling APIs.
216
+
217
+ - [ScaleLLM](https://github.com/vectorch-ai/ScaleLLM#supported-models): you can use this service to run Yi models locally with added flexibility and customization.
218
+
219
+ #### ⚙️ Quantitation
220
+
221
+ If you have limited computational capabilities, you can use Yi's quantized models as follows.
222
+
223
+ These quantized models have reduced precision and but offer increased efficiency, such as faster inference speed and smaller RAM usage.
224
+
225
+ - [TheBloke/Yi-34B-GPTQ](https://huggingface.co/TheBloke/Yi-34B-GPTQ)
226
+ - [TheBloke/Yi-34B-GGUF](https://huggingface.co/TheBloke/Yi-34B-GGUF)
227
+ - [TheBloke/Yi-34B-AWQ](https://huggingface.co/TheBloke/Yi-34B-AWQ)
228
+
229
+ #### 🛠️ Fine-tuning
230
+
231
+ If you're seeking to explore the diverse capabilities within Yi's thriving family, you can delve into Yi's fine-tuned models as below.
232
+
233
+ - [TheBloke Models](https://huggingface.co/TheBloke): this site hosts numerous fine-tuned models derived from various LLMs including Yi.
234
+
235
+ This is not an exhaustive list for Yi, but to name a few sorted on downloads:
236
+ - [TheBloke/dolphin-2_2-yi-34b-AWQ](https://huggingface.co/TheBloke/dolphin-2_2-yi-34b-AWQ)
237
+ - [TheBloke/Yi-34B-Chat-AWQ](https://huggingface.co/TheBloke/Yi-34B-Chat-AWQ)
238
+ - [TheBloke/Yi-34B-Chat-GPTQ](https://huggingface.co/TheBloke/Yi-34B-Chat-GPTQ)
239
+
240
+ - [SUSTech/SUS-Chat-34B](https://huggingface.co/SUSTech/SUS-Chat-34B): this model ranked first among all models below 70B and outperformed the twice larger deepseek-llm-67b-chat. You can check the result on the [Open LLM Leaderboard](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard).
241
+
242
+ - [OrionStarAI/OrionStar-Yi-34B-Chat-Llama](https://huggingface.co/OrionStarAI/OrionStar-Yi-34B-Chat-Llama): this model excelled beyond other models (such as GPT-4, Qwen-14B-Chat, Baichuan2-13B-Chat) in C-Eval and CMMLU evaluations on the [OpenCompass LLM Leaderboard](https://opencompass.org.cn/leaderboard-llm).
243
+
244
+ - [NousResearch/Nous-Capybara-34B](https://huggingface.co/NousResearch/Nous-Capybara-34B): this model is trained with 200K context length and 3 epochs on the Capybara dataset.
245
+
246
+ <div align="right"> [ <a href="#building-the-next-generation-of-open-source-and-bilingual-llms">Back to top ⬆️ </a> ] </div>
247
+
248
+
249
+ ## 📌 Benchmarks
250
+
251
+ - [📊 Base model performance](#-base-model-performance)
252
+ - [📊 Chat model performance](#-chat-model-performance)
253
+ - [📊 Quantized chat model performance](#-quantized-chat-model-performance)
254
+ - [⛔️ Limitations of chat model](#️-limitations-of-chat-model)
255
+
256
+ ### 📊 Base model performance
257
 
258
  | Model | MMLU | CMMLU | C-Eval | GAOKAO | BBH | Common-sense Reasoning | Reading Comprehension | Math & Code |
259
  | :------------ | :------: | :------: | :------: | :------: | :------: | :--------------------: | :-------------------: | :---------: |
 
294
  these two tasks are generally lower than the average, we believe that
295
  Falcon-180B's performance was not underestimated.
296
 
297
+ ### 📊 Chat model performance
298
 
299
  | Model | MMLU | MMLU | CMMLU | CMMLU | C-Eval(val)<sup>*</sup> | C-Eval(val)<sup>*</sup> | Truthful QA | BBH | BBH | GSM8k | GSM8k |
300
  | ----------------------- | --------- | --------- | --------- | --------- | ----------------------- | ----------------------- | ----------- | --------- | --------- | --------- | --------- |
 
316
 
317
  <strong>*</strong>: C-Eval results are evaluated on the validation datasets
318
 
319
+ ### 📊 Quantized chat model performance
320
 
321
  We also provide both 4-bit (AWQ) and 8-bit (GPTQ) quantized Yi chat models. Evaluation results on various benchmarks have shown that the quantized models have negligible losses. Additionally, they reduce the memory footprint size. After testing different configurations of prompts and generation lengths, we highly recommend following the guidelines in the memory footprint table below when selecting a device to run our models.
322
 
 
331
 
332
  Note: All the numbers in the table represent the minimum recommended memory for running models of the corresponding size.
333
 
334
+ ### ⛔️ Limitations of chat model
335
 
336
  The released chat model has undergone exclusive training using Supervised Fine-Tuning (SFT). Compared to other standard chat models, our model produces more diverse responses, making it suitable for various downstream tasks, such as creative scenarios. Furthermore, this diversity is expected to enhance the likelihood of generating higher quality responses, which will be advantageous for subsequent Reinforcement Learning (RL) training.
337
 
 
343
 
344
  To achieve more coherent and consistent responses, it is advisable to adjust generation configuration parameters such as`temperature`,`top_p`, or`top_k`. These adjustments can help in the balance between creativity and coherence in the model's outputs.
345
 
346
+ <div align="right"> [ <a href="#building-the-next-generation-of-open-source-and-bilingual-llms">Back to top ⬆️ </a> ] </div>
347
+
348
 
349
+ # 🟢 Who can use Yi?
350
 
351
+ Everyone! 🙌 ✅
352
 
353
+ - The Yi series models are free for personal usage, academic purposes, and commercial use. All usage must adhere to the [Yi Series Models Community License Agreement 2.1](https://github.com/01-ai/Yi/blob/main/MODEL_LICENSE_AGREEMENT.txt)
354
+
355
+ - For free commercial use, you only need to [complete this form](https://www.lingyiwanwu.com/yi-license) to get Yi Model Commercial License.
356
+
357
+ <div align="right"> [ <a href="#building-the-next-generation-of-open-source-and-bilingual-llms">Back to top ⬆️ </a> ] </div>
358
+
359
+ # 🟢 How to use Yi?
360
+
361
+ [1. Prepare development environment](#1-prepare-development-environment)
362
+ <br>[2. Download the model](#2-download-the-model-optional)
363
+ <br>[3. Examples](#3-examples)
364
 
365
  ### 1. Prepare development environment
366
 
 
490
  For more advanced usage, please refer to the
491
  [doc](https://github.com/01-ai/Yi/tree/main/demo).
492
 
493
+ #### 3.3 Finetune from the base model
494
 
495
  ```bash
496
  bash finetune/scripts/run_sft_Yi_6b.sh
 
543
 
544
  For more detailed explanation, please read the [doc](https://github.com/01-ai/Yi/tree/main/quantization/awq)
545
 
546
+ <div align="right"> [ <a href="#building-the-next-generation-of-open-source-and-bilingual-llms">Back to top ⬆️ </a> ] </div>
 
 
547
 
548
+ # 🟢 Misc.
 
 
549
 
550
+ ### 📡 Disclaimer
551
 
552
  We use data compliance checking algorithms during the training process, to
553
  ensure the compliance of the trained model to the best of our ability. Due to
 
558
  resulting from misuse, misguidance, illegal usage, and related misinformation,
559
  as well as any associated data security concerns.
560
 
561
+ <div align="right"> [ <a href="#building-the-next-generation-of-open-source-and-bilingual-llms">Back to top ⬆️ </a> ] </div>
562
+
563
+
564
+ ### 🪪 License
565
 
566
  The source code in this repo is licensed under the [Apache 2.0
567
  license](https://github.com/01-ai/Yi/blob/main/LICENSE). The Yi series models
568
  are fully open for academic research and free commercial usage with permission
569
+ via applications. All usage must adhere to the [Yi Series Models Community License Agreement 2.1](https://github.com/01-ai/Yi/blob/main/MODEL_LICENSE_AGREEMENT.txt).
570
+ For free commercial use, you only need to send an email to [get official commercial permission](https://www.lingyiwanwu.com/yi-license).
571
+
572
+ <div align="right"> [ <a href="#building-the-next-generation-of-open-source-and-bilingual-llms">Back to top ⬆️ </a> ] </div>