Gaivoronsky commited on
Commit
fcaea19
1 Parent(s): 75a98eb

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +12 -1
README.md CHANGED
@@ -11,7 +11,7 @@ pipeline_tag: text-generation
11
  ---
12
  This is a generative model converted to fp16 format based on [IlyaGusev/saiga_mistral_7b_lora](https://huggingface.co/IlyaGusev/saiga_mistral_7b_lora)
13
 
14
- Install vLLM
15
  ```bash
16
  pip install vllm
17
  ```
@@ -19,4 +19,15 @@ pip install vllm
19
  Start server:
20
  ```bash
21
  python -u -m vllm.entrypoints.openai.api_server --host 0.0.0.0 --model Gaivoronsky/Mistral-7B-Saiga
 
 
 
 
 
 
 
 
 
 
 
22
  ```
 
11
  ---
12
  This is a generative model converted to fp16 format based on [IlyaGusev/saiga_mistral_7b_lora](https://huggingface.co/IlyaGusev/saiga_mistral_7b_lora)
13
 
14
+ Install vLLM:
15
  ```bash
16
  pip install vllm
17
  ```
 
19
  Start server:
20
  ```bash
21
  python -u -m vllm.entrypoints.openai.api_server --host 0.0.0.0 --model Gaivoronsky/Mistral-7B-Saiga
22
+ ```
23
+
24
+ Client:
25
+ ```python
26
+ import openai
27
+ response = openai.ChatCompletion.create(
28
+ model="Gaivoronsky/Mistral-7B-Saiga",
29
+ messages=[{"role": "user", "content": 'Привет'}],
30
+ max_tokens=512,
31
+ )
32
+ response['choices'][0]['message']['content']
33
  ```