Update README.md
Browse files
README.md
CHANGED
@@ -11,9 +11,13 @@ IQ2-GGUF quants of [sophosympatheia/Aurora-Nights-70B-v1.0](https://huggingface.
|
|
11 |
|
12 |
Unlike regular GGUF quants this uses important matrix similar to Quip# to keep the quant from degrading too much even at 2bpw allowing you to run larger models on less powerful machines.
|
13 |
|
14 |
-
***NOTE:***
|
|
|
|
|
|
|
|
|
|
|
15 |
|
16 |
-
[More info](https://github.com/ggerganov/llama.cpp/pull/4897)
|
17 |
|
18 |
|
19 |
# Models
|
|
|
11 |
|
12 |
Unlike regular GGUF quants this uses important matrix similar to Quip# to keep the quant from degrading too much even at 2bpw allowing you to run larger models on less powerful machines.
|
13 |
|
14 |
+
***NOTE:*** Currently you will need experimental branches of Koboldcpp or Ooba for this to work.
|
15 |
+
- Nexesenex have compiled Windows binaries [HERE](https://github.com/Nexesenex/kobold.cpp/releases/tag/v1.55.1_b1842)
|
16 |
+
- [llamacpp_0.2.29 branch](https://github.com/oobabooga/text-generation-webui/tree/llamacpp_0.2.29) of Ooba also works
|
17 |
+
|
18 |
+
|
19 |
+
[More info about IQ2](https://github.com/ggerganov/llama.cpp/pull/4897)
|
20 |
|
|
|
21 |
|
22 |
|
23 |
# Models
|