Update README.md
Browse files
README.md
CHANGED
@@ -88,7 +88,7 @@ Thai language multiple choice exams, Test on unseen test set, Zero-shot learning
|
|
88 |
- E-mail: kobkrit@aieat.or.th
|
89 |
|
90 |
## Prompt Format
|
91 |
-
Prompt format is based on
|
92 |
```
|
93 |
<|im_start|>system\n{sytem_prompt}<|im_end|>\n<|im_start|>user\n{prompt}<|im_end|>\n<|im_start|>assistant\n
|
94 |
```
|
@@ -175,6 +175,8 @@ response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
|
|
175 |
```bash
|
176 |
vllm serve openthaigpt/openthaigpt1.5-72b-instruct --tensor-parallel-size 4
|
177 |
```
|
|
|
|
|
178 |
3. Run inference (CURL example)
|
179 |
```bash
|
180 |
curl -X POST 'http://127.0.0.1:8000/v1/completions' \
|
|
|
88 |
- E-mail: kobkrit@aieat.or.th
|
89 |
|
90 |
## Prompt Format
|
91 |
+
Prompt format is based on ChatML.
|
92 |
```
|
93 |
<|im_start|>system\n{sytem_prompt}<|im_end|>\n<|im_start|>user\n{prompt}<|im_end|>\n<|im_start|>assistant\n
|
94 |
```
|
|
|
175 |
```bash
|
176 |
vllm serve openthaigpt/openthaigpt1.5-72b-instruct --tensor-parallel-size 4
|
177 |
```
|
178 |
+
* Note, change ``--tensor-parallel-size 4`` to the amount of available GPU cards.
|
179 |
+
|
180 |
3. Run inference (CURL example)
|
181 |
```bash
|
182 |
curl -X POST 'http://127.0.0.1:8000/v1/completions' \
|