nisten commited on
Commit
4ad5e94
1 Parent(s): 12c81ee

Fixed 5am spelling mistakes

Browse files
Files changed (1) hide show
  1. README.md +5 -4
README.md CHANGED
@@ -4,8 +4,8 @@ base_model: [deepseek-ai/DeepSeek-Coder-V2-Instruct]
4
 
5
  #### Custom quantizations of deepseek-coder-v2-instruct optimized for cpu inference.
6
 
7
- ### Theis ones uses GGML TYPE IQ_4_XS in combination with q8_0 so it runs fast with minimal loss and takes advantage of int8 optimizations on most nevwer server cpus.
8
- ### While it required custom code to make, it is standard compatible with plain llama.cpp from github or just search nisten in lmstudio.
9
 
10
  >[!TIP]
11
  >The following 4bit version is the one I use myself, it gets 17tps on 64 arm cores.
@@ -45,8 +45,9 @@ aria2c -x 8 -o deepseek_coder_v2_cpu_iq4xm.gguf-00003-of-00004.gguf \
45
  aria2c -x 8 -o deepseek_coder_v2_cpu_iq4xm.gguf-00004-of-00004.gguf \
46
  https://huggingface.co/nisten/deepseek-coder-v2-inst-cpu-optimized-gguf/resolve/main/deepseek_coder_v2_cpu_iq4xm.gguf-00004-of-00004.gguf
47
  ```
 
48
  >[!TIP]
49
- > ### And for downloading the Q8_0 version converted in the most lossless way possible from hf bf16 model:
50
  >
51
  ```verilog
52
 
@@ -69,6 +70,6 @@ aria2c -x 8 -o deepseek_coder_v2_cpu_q8_0-00006-of-00006.gguf \
69
  https://huggingface.co/nisten/deepseek-coder-v2-inst-cpu-optimized-gguf/resolve/main/deepseek_coder_v2_cpu_q8_0-00006-of-00006.gguf
70
  ```
71
 
72
- Enjoy and remembeter to accelerate!
73
 
74
  -Nisten
 
4
 
5
  #### Custom quantizations of deepseek-coder-v2-instruct optimized for cpu inference.
6
 
7
+ ### This iq4xm one uses GGML TYPE IQ_4_XS 4bit in combination with q8_0 bit so it runs fast with minimal loss and takes advantage of int8 optimizations on most newer server cpus.
8
+ ### While it required custom code to make, it is compatible with standard llama.cpp from github or just search nisten in lmstudio.
9
 
10
  >[!TIP]
11
  >The following 4bit version is the one I use myself, it gets 17tps on 64 arm cores.
 
45
  aria2c -x 8 -o deepseek_coder_v2_cpu_iq4xm.gguf-00004-of-00004.gguf \
46
  https://huggingface.co/nisten/deepseek-coder-v2-inst-cpu-optimized-gguf/resolve/main/deepseek_coder_v2_cpu_iq4xm.gguf-00004-of-00004.gguf
47
  ```
48
+
49
  >[!TIP]
50
+ > ### And for downloading the Q8_0 version converted in the most lossless way possible from hf bf16 download these:
51
  >
52
  ```verilog
53
 
 
70
  https://huggingface.co/nisten/deepseek-coder-v2-inst-cpu-optimized-gguf/resolve/main/deepseek_coder_v2_cpu_q8_0-00006-of-00006.gguf
71
  ```
72
 
73
+ Enjoy and remember to accelerate!
74
 
75
  -Nisten