macadeliccc commited on
Commit
407a023
1 Parent(s): 043d081

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +9 -0
README.md CHANGED
@@ -1,3 +1,12 @@
1
  ---
2
  license: apache-2.0
 
3
  ---
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: apache-2.0
3
+ base_model: macadeliccc/laser-dolphin-mixtral-4x7b-dpo
4
  ---
5
+
6
+ ## OpenAI compatible endpoint using VLLM
7
+
8
+ Runs well on 4090
9
+
10
+ ```
11
+ python -m vllm.entrypoints.openai.api_server --model macadeliccc/laser-dolphin-mixtral-4x7b-dpo-AWQ --max-model-len 25000
12
+ ```