Update README.md
Browse files
README.md
CHANGED
@@ -90,9 +90,9 @@ python -m vllm.entrypoints.openai.api_server --model astronomer-io/Llama-3-8B-In
|
|
90 |
```
|
91 |
For the non-stop token generation bug, make sure to send requests with `stop_token_ids":[128001, 128009]` to vLLM endpoint
|
92 |
Example:
|
93 |
-
```
|
94 |
{
|
95 |
-
"model": "Llama-3-8B-Instruct-GPTQ-8-Bit",
|
96 |
"messages": [
|
97 |
{"role": "system", "content": "You are a helpful assistant."},
|
98 |
{"role": "user", "content": "Who created Llama 3?"}
|
|
|
90 |
```
|
91 |
For the non-stop token generation bug, make sure to send requests with `stop_token_ids":[128001, 128009]` to vLLM endpoint
|
92 |
Example:
|
93 |
+
```json
|
94 |
{
|
95 |
+
"model": "astronomer-io/Llama-3-8B-Instruct-GPTQ-8-Bit",
|
96 |
"messages": [
|
97 |
{"role": "system", "content": "You are a helpful assistant."},
|
98 |
{"role": "user", "content": "Who created Llama 3?"}
|