danielhanchen commited on
Commit
17ec705
·
verified ·
1 Parent(s): ba27025

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +17 -14
README.md CHANGED
@@ -18,14 +18,16 @@ tags:
18
  Or you can view more detailed instructions here: [unsloth.ai/blog/deepseek-r1](https://unsloth.ai/blog/deepseek-r1)
19
  1. Do not forget about `<|User|>` and `<|Assistant|>` tokens! - Or use a chat template formatter
20
  2. Obtain the latest `llama.cpp` at https://github.com/ggerganov/llama.cpp
21
- 3. Example with Q5_0 K quantized cache **Notice -no-cnv disables auto conversation mode**
22
  ```bash
23
- ./llama.cpp/llama-cli \
24
- --model unsloth/DeepSeek-R1-GGUF/DeepSeek-R1-Q2_K_XS.gguf \
25
- --cache-type-k q5_0 \
26
- --threads 16 \
27
- --prompt '<|User|>What is 1+1?<|Assistant|>' \
28
- -no-cnv
 
 
29
  ```
30
  Example output:
31
 
@@ -40,13 +42,14 @@ Or you can view more detailed instructions here: [unsloth.ai/blog/deepseek-r1](h
40
 
41
  4. If you have a GPU (RTX 4090 for example) with 24GB, you can offload multiple layers to the GPU for faster processing. If you have multiple GPUs, you can probably offload more layers.
42
  ```bash
43
- ./llama.cpp/llama-cli \
44
- --model unsloth/DeepSeek-R1-GGUF/DeepSeek-R1-Q2_K_XS.gguf \
45
- --cache-type-k q5_0 \
46
- --threads 16
47
- --prompt '<|User|>What is 1+1?<|Assistant|>'
48
- --n-gpu-layers 20 \
49
- -no-cnv
 
50
  ```
51
  5. If you want to merge the weights together, use this script:
52
  ```
 
18
  Or you can view more detailed instructions here: [unsloth.ai/blog/deepseek-r1](https://unsloth.ai/blog/deepseek-r1)
19
  1. Do not forget about `<|User|>` and `<|Assistant|>` tokens! - Or use a chat template formatter
20
  2. Obtain the latest `llama.cpp` at https://github.com/ggerganov/llama.cpp
21
+ 3. Example with Q4_0 K quantized cache **Notice -no-cnv disables auto conversation mode**
22
  ```bash
23
+ ./llama.cpp/llama-cli \
24
+ --model DeepSeek-R1-UD-IQ1_S/DeepSeek-R1-UD-IQ1_S-00001-of-00003.gguf \
25
+ --cache-type-k q4_0 \
26
+ --threads 12 -no-cnv --prio 2 \
27
+ --temp 0.6 \
28
+ --ctx-size 8192 \
29
+ --seed 3407 \
30
+ --prompt "<|User|>Create a Flappy Bird game in Python.<|Assistant|>"
31
  ```
32
  Example output:
33
 
 
42
 
43
  4. If you have a GPU (RTX 4090 for example) with 24GB, you can offload multiple layers to the GPU for faster processing. If you have multiple GPUs, you can probably offload more layers.
44
  ```bash
45
+ ./llama.cpp/llama-cli \
46
+ --model DeepSeek-R1-UD-IQ1_S/DeepSeek-R1-UD-IQ1_S-00001-of-00003.gguf \
47
+ --cache-type-k q4_0 \
48
+ --threads 12 -no-cnv --n-gpu-layers 7 --prio 2 \
49
+ --temp 0.6 \
50
+ --ctx-size 8192 \
51
+ --seed 3407 \
52
+ --prompt "<|User|>Create a Flappy Bird game in Python.<|Assistant|>"
53
  ```
54
  5. If you want to merge the weights together, use this script:
55
  ```