Update README.md
Browse files
README.md
CHANGED
@@ -18,14 +18,16 @@ tags:
|
|
18 |
Or you can view more detailed instructions here: [unsloth.ai/blog/deepseek-r1](https://unsloth.ai/blog/deepseek-r1)
|
19 |
1. Do not forget about `<|User|>` and `<|Assistant|>` tokens! - Or use a chat template formatter
|
20 |
2. Obtain the latest `llama.cpp` at https://github.com/ggerganov/llama.cpp
|
21 |
-
3. Example with
|
22 |
```bash
|
23 |
-
|
24 |
-
|
25 |
-
|
26 |
-
|
27 |
-
|
28 |
-
|
|
|
|
|
29 |
```
|
30 |
Example output:
|
31 |
|
@@ -40,13 +42,14 @@ Or you can view more detailed instructions here: [unsloth.ai/blog/deepseek-r1](h
|
|
40 |
|
41 |
4. If you have a GPU (RTX 4090 for example) with 24GB, you can offload multiple layers to the GPU for faster processing. If you have multiple GPUs, you can probably offload more layers.
|
42 |
```bash
|
43 |
-
|
44 |
-
|
45 |
-
|
46 |
-
|
47 |
-
|
48 |
-
|
49 |
-
|
|
|
50 |
```
|
51 |
5. If you want to merge the weights together, use this script:
|
52 |
```
|
|
|
18 |
Or you can view more detailed instructions here: [unsloth.ai/blog/deepseek-r1](https://unsloth.ai/blog/deepseek-r1)
|
19 |
1. Do not forget about `<|User|>` and `<|Assistant|>` tokens! - Or use a chat template formatter
|
20 |
2. Obtain the latest `llama.cpp` at https://github.com/ggerganov/llama.cpp
|
21 |
+
3. Example with Q4_0 K quantized cache **Notice -no-cnv disables auto conversation mode**
|
22 |
```bash
|
23 |
+
./llama.cpp/llama-cli \
|
24 |
+
--model DeepSeek-R1-UD-IQ1_S/DeepSeek-R1-UD-IQ1_S-00001-of-00003.gguf \
|
25 |
+
--cache-type-k q4_0 \
|
26 |
+
--threads 12 -no-cnv --prio 2 \
|
27 |
+
--temp 0.6 \
|
28 |
+
--ctx-size 8192 \
|
29 |
+
--seed 3407 \
|
30 |
+
--prompt "<|User|>Create a Flappy Bird game in Python.<|Assistant|>"
|
31 |
```
|
32 |
Example output:
|
33 |
|
|
|
42 |
|
43 |
4. If you have a GPU (RTX 4090 for example) with 24GB, you can offload multiple layers to the GPU for faster processing. If you have multiple GPUs, you can probably offload more layers.
|
44 |
```bash
|
45 |
+
./llama.cpp/llama-cli \
|
46 |
+
--model DeepSeek-R1-UD-IQ1_S/DeepSeek-R1-UD-IQ1_S-00001-of-00003.gguf \
|
47 |
+
--cache-type-k q4_0 \
|
48 |
+
--threads 12 -no-cnv --n-gpu-layers 7 --prio 2 \
|
49 |
+
--temp 0.6 \
|
50 |
+
--ctx-size 8192 \
|
51 |
+
--seed 3407 \
|
52 |
+
--prompt "<|User|>Create a Flappy Bird game in Python.<|Assistant|>"
|
53 |
```
|
54 |
5. If you want to merge the weights together, use this script:
|
55 |
```
|