kobkrit commited on
Commit
49fd75c
โ€ข
1 Parent(s): dcd8202

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +3 -1
README.md CHANGED
@@ -88,7 +88,7 @@ Thai language multiple choice exams, Test on unseen test set, Zero-shot learning
88
  - E-mail: kobkrit@aieat.or.th
89
 
90
  ## Prompt Format
91
- Prompt format is based on Llama2 with a small modification (Adding "###" to specify the context part)
92
  ```
93
  <|im_start|>system\n{sytem_prompt}<|im_end|>\n<|im_start|>user\n{prompt}<|im_end|>\n<|im_start|>assistant\n
94
  ```
@@ -175,6 +175,8 @@ response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
175
  ```bash
176
  vllm serve openthaigpt/openthaigpt1.5-72b-instruct --tensor-parallel-size 4
177
  ```
 
 
178
  3. Run inference (CURL example)
179
  ```bash
180
  curl -X POST 'http://127.0.0.1:8000/v1/completions' \
 
88
  - E-mail: kobkrit@aieat.or.th
89
 
90
  ## Prompt Format
91
+ Prompt format is based on ChatML.
92
  ```
93
  <|im_start|>system\n{sytem_prompt}<|im_end|>\n<|im_start|>user\n{prompt}<|im_end|>\n<|im_start|>assistant\n
94
  ```
 
175
  ```bash
176
  vllm serve openthaigpt/openthaigpt1.5-72b-instruct --tensor-parallel-size 4
177
  ```
178
+ * Note, change ``--tensor-parallel-size 4`` to the amount of available GPU cards.
179
+
180
  3. Run inference (CURL example)
181
  ```bash
182
  curl -X POST 'http://127.0.0.1:8000/v1/completions' \