Update README.md
Browse files
README.md
CHANGED
@@ -21,8 +21,6 @@ This repo contains GGUF format model files for [SpeakLeash](https://speakleash.o
|
|
21 |
|
22 |
<b><u>DISCLAIMER: Be aware that quantised models show reduced response quality and possible hallucinations!</u></b><br>
|
23 |
|
24 |
-
Simple Colab notebook for testing: https://colab.research.google.com/drive/1Al9glPVCuOXbtDsks8cMcuzkuu8YDzpg?usp=sharing
|
25 |
-
|
26 |
### Available quantization formats:
|
27 |
* **q4_k_m:** Uses Q6_K for half of the attention.wv and feed_forward.w2 tensors, else Q4_K
|
28 |
* **q5_k_m:** Uses Q6_K for half of the attention.wv and feed_forward.w2 tensors, else Q5_K
|
|
|
21 |
|
22 |
<b><u>DISCLAIMER: Be aware that quantised models show reduced response quality and possible hallucinations!</u></b><br>
|
23 |
|
|
|
|
|
24 |
### Available quantization formats:
|
25 |
* **q4_k_m:** Uses Q6_K for half of the attention.wv and feed_forward.w2 tensors, else Q4_K
|
26 |
* **q5_k_m:** Uses Q6_K for half of the attention.wv and feed_forward.w2 tensors, else Q5_K
|