File size: 275 Bytes
e8f9762 407a023 e8f9762 407a023 |
1 2 3 4 5 6 7 8 9 10 11 12 |
---
license: apache-2.0
base_model: macadeliccc/laser-dolphin-mixtral-4x7b-dpo
---
## OpenAI compatible endpoint using VLLM
Runs well on 4090
```
python -m vllm.entrypoints.openai.api_server --model macadeliccc/laser-dolphin-mixtral-4x7b-dpo-AWQ --max-model-len 25000
``` |