yi-01-ai commited on
Commit
edb8bb8
1 Parent(s): cb47ed8

Auto Sync from git://github.com/01-ai/Yi.git/commit/a7cc760cb87e7c21a330ddca0d14c14565e72b14

Browse files
Files changed (1) hide show
  1. README.md +39 -11
README.md CHANGED
@@ -81,7 +81,9 @@ pipeline_tag: text-generation
81
  - [🟢 How to use Yi?](#-how-to-use-yi)
82
  - [Quick start](#quick-start)
83
  - [Choose your path](#choose-your-parth)
84
- - [Tutorial](#tutorial)
 
 
85
  - [Fine tune](#fine-tune)
86
  - [Quantization](#quantization)
87
  - [Deployment](https://github.com/01-ai/Yi/blob/main/docs/deployment.md)
@@ -231,7 +233,9 @@ sequence length and can be extended to 32K during inference time.
231
 
232
  - [Quick start](#quick-start)
233
  - [Choose your path](#choose-your-parth)
234
- - [Tutorial](#tutorial)
 
 
235
  - [Fine tune](#fine-tune)
236
  - [Quantization](#quantization)
237
  - [Deployment](https://github.com/01-ai/Yi/blob/main/docs/deployment.md)
@@ -252,7 +256,7 @@ Select one of the following paths to begin your journey with Yi!
252
  If you prefer to deploy Yi models locally,
253
 
254
  - 🙋‍♀️ and you have **sufficient** resources (for example, NVIDIA A800 80GB), you can choose one of the following methods:
255
- - [pip](#tutorial)
256
  - [Docker](https://github.com/01-ai/Yi/blob/main/docs/README_legacy.md#11-docker)
257
  - [conda-lock](https://github.com/01-ai/Yi/blob/main/docs/README_legacy.md#12-local-development-environment)
258
 
@@ -290,18 +294,18 @@ If you want to chat with Yi with more customizable options (e.g., system prompt,
290
  - [Yi-34B-Chat](https://platform.lingyiwanwu.com/) (Yi official beta)
291
  - Access is available through a whitelist. Welcome to apply (fill out a form in [English](https://cn.mikecrm.com/l91ODJf) or [Chinese](https://cn.mikecrm.com/gnEZjiQ)).
292
 
293
- ## Tutorial
294
 
295
  This tutorial guides you through every step of running Yi (Yi-34B-Chat) locally and then performing inference.
296
 
297
- ### Step 0: Prerequistes
298
 
299
  - This tutorial assumes you are running the **Yi-34B-Chat** with an **A800 (80G)** GPU.
300
  - For detailed deployment requirements to run Yi models, see [hardware requirements]( https://github.com/01-ai/Yi/blob/main/docs/deployment.md).
301
 
302
  - Make sure Python 3.10 or later version is installed.
303
 
304
- ### Step 1: Prepare environment
305
 
306
  To set up the environment and install the required packages, execute the following command.
307
 
@@ -311,7 +315,7 @@ cd yi
311
  pip install -r requirements.txt
312
  ```
313
 
314
- ### Step 2: Download Yi model
315
 
316
  You can download the weights and tokenizer of Yi models from the following sources:
317
 
@@ -319,11 +323,11 @@ You can download the weights and tokenizer of Yi models from the following sourc
319
  - [ModelScope](https://www.modelscope.cn/organization/01ai/)
320
  - [WiseModel](https://wisemodel.cn/organization/01.AI)
321
 
322
- ### Step 3: Perform inference
323
 
324
  You can perform inference with Yi chat or base models as below.
325
 
326
- #### Perform inference with Yi chat model
327
 
328
  1. Create a file named `quick_start.py` and copy the following content to it.
329
 
@@ -366,7 +370,7 @@ You can perform inference with Yi chat or base models as below.
366
  Hello! How can I assist you today?
367
  ```
368
 
369
- #### Perform inference with Yi base model
370
 
371
  The steps are similar to [Run Yi chat model](#run-yi-chat-model).
372
 
@@ -390,6 +394,30 @@ Then you can see an output similar to the one below. 🥳
390
 
391
  </details>
392
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
393
  ### Finetuning
394
 
395
  ```bash
@@ -685,4 +713,4 @@ are fully open for academic research and free commercial usage with permission
685
  via applications. All usage must adhere to the [Yi Series Models Community License Agreement 2.1](https://github.com/01-ai/Yi/blob/main/MODEL_LICENSE_AGREEMENT.txt).
686
  For free commercial use, you only need to send an email to [get official commercial permission](https://www.lingyiwanwu.com/yi-license).
687
 
688
- <div align="right"> [ <a href="#building-the-next-generation-of-open-source-and-bilingual-llms">Back to top ⬆️ </a> ] </div>
 
81
  - [🟢 How to use Yi?](#-how-to-use-yi)
82
  - [Quick start](#quick-start)
83
  - [Choose your path](#choose-your-parth)
84
+ - [pip](#pip)
85
+ - [llama.cpp](https://github.com/01-ai/Yi/blob/main/docs/yi_llama.cpp.md)
86
+ - [Web demo](#web-demo)
87
  - [Fine tune](#fine-tune)
88
  - [Quantization](#quantization)
89
  - [Deployment](https://github.com/01-ai/Yi/blob/main/docs/deployment.md)
 
233
 
234
  - [Quick start](#quick-start)
235
  - [Choose your path](#choose-your-parth)
236
+ - [pip](#pip)
237
+ - [llama.cpp](https://github.com/01-ai/Yi/blob/main/docs/yi_llama.cpp.md)
238
+ - [Web demo](#web-demo)
239
  - [Fine tune](#fine-tune)
240
  - [Quantization](#quantization)
241
  - [Deployment](https://github.com/01-ai/Yi/blob/main/docs/deployment.md)
 
256
  If you prefer to deploy Yi models locally,
257
 
258
  - 🙋‍♀️ and you have **sufficient** resources (for example, NVIDIA A800 80GB), you can choose one of the following methods:
259
+ - [pip](#pip)
260
  - [Docker](https://github.com/01-ai/Yi/blob/main/docs/README_legacy.md#11-docker)
261
  - [conda-lock](https://github.com/01-ai/Yi/blob/main/docs/README_legacy.md#12-local-development-environment)
262
 
 
294
  - [Yi-34B-Chat](https://platform.lingyiwanwu.com/) (Yi official beta)
295
  - Access is available through a whitelist. Welcome to apply (fill out a form in [English](https://cn.mikecrm.com/l91ODJf) or [Chinese](https://cn.mikecrm.com/gnEZjiQ)).
296
 
297
+ ### pip
298
 
299
  This tutorial guides you through every step of running Yi (Yi-34B-Chat) locally and then performing inference.
300
 
301
+ #### Step 0: Prerequistes
302
 
303
  - This tutorial assumes you are running the **Yi-34B-Chat** with an **A800 (80G)** GPU.
304
  - For detailed deployment requirements to run Yi models, see [hardware requirements]( https://github.com/01-ai/Yi/blob/main/docs/deployment.md).
305
 
306
  - Make sure Python 3.10 or later version is installed.
307
 
308
+ #### Step 1: Prepare your environment
309
 
310
  To set up the environment and install the required packages, execute the following command.
311
 
 
315
  pip install -r requirements.txt
316
  ```
317
 
318
+ #### Step 2: Download the Yi model
319
 
320
  You can download the weights and tokenizer of Yi models from the following sources:
321
 
 
323
  - [ModelScope](https://www.modelscope.cn/organization/01ai/)
324
  - [WiseModel](https://wisemodel.cn/organization/01.AI)
325
 
326
+ #### Step 3: Perform inference
327
 
328
  You can perform inference with Yi chat or base models as below.
329
 
330
+ ##### Perform inference with Yi chat model
331
 
332
  1. Create a file named `quick_start.py` and copy the following content to it.
333
 
 
370
  Hello! How can I assist you today?
371
  ```
372
 
373
+ ##### Perform inference with Yi base model
374
 
375
  The steps are similar to [Run Yi chat model](#run-yi-chat-model).
376
 
 
394
 
395
  </details>
396
 
397
+ ### Run Yi with llama.cpp
398
+
399
+ If you have limited resources, you can try [llama.cpp](https://github.com/ggerganov/llama.cpp) or [ollama.cpp](https://ollama.ai/) (especially for Chinese users) to run Yi models in a few minutes locally.
400
+
401
+ For a step-by-step tutorial,, see [Run Yi with llama.cpp](https://github.com/01-ai/Yi/edit/main/docs/yi_llama.cpp.md).
402
+
403
+ ### Web demo
404
+
405
+ You can build a web UI demo for Yi **chat** models (note that Yi base models are not supported in this senario).
406
+
407
+ [Step 1: Prepare your environment](#step-1-prepare-your-environment).
408
+
409
+ [Step 2: Download the Yi model](#step-2-download-the-yi-model).
410
+
411
+ Step 3. To start a web service locally, run the following command.
412
+
413
+ ```bash
414
+ python demo/web_demo.py --checkpoint-path <your-model-path>
415
+ ```
416
+
417
+ You can access the web UI by entering the address provided in the console into your browser.
418
+
419
+ ![Quick start - web demo](./assets/img/yi_34b_chat_web_demo.gif)
420
+
421
  ### Finetuning
422
 
423
  ```bash
 
713
  via applications. All usage must adhere to the [Yi Series Models Community License Agreement 2.1](https://github.com/01-ai/Yi/blob/main/MODEL_LICENSE_AGREEMENT.txt).
714
  For free commercial use, you only need to send an email to [get official commercial permission](https://www.lingyiwanwu.com/yi-license).
715
 
716
+ <div align="right"> [ <a href="#building-the-next-generation-of-open-source-and-bilingual-llms">Back to top ⬆️ </a> ] </div>