feihu.hf
commited on
Commit
•
d30e8dd
1
Parent(s):
5baac52
update README.md
Browse files
README.md
CHANGED
@@ -79,7 +79,13 @@ To handle extensive inputs exceeding 32,768 tokens, we utilize [YARN](https://ar
|
|
79 |
|
80 |
For deployment, we recommend using vLLM. You can enable the long-context capabilities by following these steps:
|
81 |
|
82 |
-
1. **Install vLLM**:
|
|
|
|
|
|
|
|
|
|
|
|
|
83 |
|
84 |
2. **Configure Model Settings**: After downloading the model weights, modify the `config.json` file by including the below snippet:
|
85 |
```json
|
|
|
79 |
|
80 |
For deployment, we recommend using vLLM. You can enable the long-context capabilities by following these steps:
|
81 |
|
82 |
+
1. **Install vLLM**: You can install vLLM by running the following command.
|
83 |
+
|
84 |
+
```bash
|
85 |
+
pip install "vllm>=0.4.3"
|
86 |
+
```
|
87 |
+
|
88 |
+
Or you can install vLLM from [source](https://github.com/vllm-project/vllm/).
|
89 |
|
90 |
2. **Configure Model Settings**: After downloading the model weights, modify the `config.json` file by including the below snippet:
|
91 |
```json
|