| license: apache-2.0 | |
| base_model: macadeliccc/laser-dolphin-mixtral-4x7b-dpo | |
| ## OpenAI compatible endpoint using VLLM | |
| Runs well on 4090 | |
| ``` | |
| python -m vllm.entrypoints.openai.api_server --model macadeliccc/laser-dolphin-mixtral-4x7b-dpo-AWQ --max-model-len 25000 | |
| ``` |