Update README.md
Browse files
README.md
CHANGED
@@ -11,7 +11,7 @@ license: other
|
|
11 |
<p><a href="https://discord.gg/UBgz4VXf">Chat & support: my new Discord server</a></p>
|
12 |
</div>
|
13 |
<div style="display: flex; flex-direction: column; align-items: flex-end;">
|
14 |
-
<p><a href="https://www.patreon.com/TheBlokeAI">Want to contribute? Patreon
|
15 |
</div>
|
16 |
</div>
|
17 |
|
@@ -47,6 +47,14 @@ I have quantised the GGML files in this repo with the latest version. Therefore
|
|
47 |
| samantha-33B.ggmlv3.q5_1.bin | q5_1 | 5 | 24.40 GB | 26.90 GB | 5-bit. Even higher accuracy, resource usage and slower inference. |
|
48 |
| samantha-33B.ggmlv3.q8_0.bin | q8_0 | 8 | 34.56 GB | 37.06 GB | 8-bit. Almost indistinguishable from float16. Huge resource use and slow. Not recommended for normal use. |
|
49 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
50 |
|
51 |
## How to run in `llama.cpp`
|
52 |
|
|
|
11 |
<p><a href="https://discord.gg/UBgz4VXf">Chat & support: my new Discord server</a></p>
|
12 |
</div>
|
13 |
<div style="display: flex; flex-direction: column; align-items: flex-end;">
|
14 |
+
<p><a href="https://www.patreon.com/TheBlokeAI">Want to contribute? TheBloke's Patreon page</a></p>
|
15 |
</div>
|
16 |
</div>
|
17 |
|
|
|
47 |
| samantha-33B.ggmlv3.q5_1.bin | q5_1 | 5 | 24.40 GB | 26.90 GB | 5-bit. Even higher accuracy, resource usage and slower inference. |
|
48 |
| samantha-33B.ggmlv3.q8_0.bin | q8_0 | 8 | 34.56 GB | 37.06 GB | 8-bit. Almost indistinguishable from float16. Huge resource use and slow. Not recommended for normal use. |
|
49 |
|
50 |
+
## Prompt template example
|
51 |
+
|
52 |
+
```
|
53 |
+
You are Samantha, a sentient AI.
|
54 |
+
|
55 |
+
USER: <prompt>
|
56 |
+
ASSISTANT:
|
57 |
+
```
|
58 |
|
59 |
## How to run in `llama.cpp`
|
60 |
|