Commit
•
dd7908b
1
Parent(s):
6125ba4
Update README.md
Browse filesAdd the execution commands
README.md
CHANGED
@@ -8,20 +8,39 @@ model_name: Openchat 3.5 0106
|
|
8 |
model_type: mistral
|
9 |
pipeline_tag: text-generation
|
10 |
quantized_by: Second State Inc.
|
|
|
|
|
11 |
---
|
12 |
|
13 |
-
# OpenChat-3.5-0106
|
14 |
|
15 |
## Original Model
|
16 |
|
17 |
[openchat/openchat-3.5-0106](https://huggingface.co/openchat/openchat-3.5-0106)
|
18 |
|
19 |
-
##
|
20 |
|
21 |
-
-
|
22 |
|
23 |
-
- Prompt
|
24 |
|
25 |
-
|
26 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
27 |
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
8 |
model_type: mistral
|
9 |
pipeline_tag: text-generation
|
10 |
quantized_by: Second State Inc.
|
11 |
+
tags:
|
12 |
+
- openchat
|
13 |
---
|
14 |
|
15 |
+
# OpenChat-3.5-0106-GGUF
|
16 |
|
17 |
## Original Model
|
18 |
|
19 |
[openchat/openchat-3.5-0106](https://huggingface.co/openchat/openchat-3.5-0106)
|
20 |
|
21 |
+
## Run with LlamaEdge
|
22 |
|
23 |
+
- LlamaEdge version: [v0.2.4](https://github.com/second-state/LlamaEdge/releases/tag/0.2.4)
|
24 |
|
25 |
+
- Prompt template
|
26 |
|
27 |
+
- Prompt type: `openchat`
|
28 |
+
|
29 |
+
- Prompt string
|
30 |
+
|
31 |
+
```text
|
32 |
+
GPT4 User: {prompt}<|end_of_turn|>GPT4 Assistant:
|
33 |
+
```
|
34 |
+
|
35 |
+
- Run as LlamaEdge service
|
36 |
+
|
37 |
+
```bash
|
38 |
+
wasmedge --dir .:. --nn-preload default:GGML:AUTO:openchat-3.5-0106-Q5_K_M.gguf llama-api-server.wasm -p openchat -r '<|end_of_turn|>'
|
39 |
```
|
40 |
+
|
41 |
+
- Run as LlamaEdge command app
|
42 |
+
|
43 |
+
```bash
|
44 |
+
wasmedge --dir .:. --nn-preload default:GGML:AUTO:openchat-3.5-0106-Q5_K_M.gguf llama-chat.wasm -p openchat -r '<|end_of_turn|>'
|
45 |
+
```
|
46 |
+
|