File size: 1,369 Bytes
2167c42 81b06d7 2167c42 e9c9880 2167c42 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 |
---
license: cc-by-nc-4.0
language:
- en
---
# WinterGoddess-1.4x-70B-L2 IQ2-GGUF
## Description
IQ2-GGUF quants of [Sao10K/WinterGoddess-1.4x-70B-L2](https://huggingface.co/Sao10K/WinterGoddess-1.4x-70B-L2)
Unlike regular GGUF quants this uses important matrix similar to Quip# to keep the quant from degrading too much even at 2bpw allowing you to run larger models on less powerful machines.
***NOTE:*** Currently you will need experimental branches of Koboldcpp or Ooba for this to work.
- Nexesenex have compiled Windows binaries [HERE](https://github.com/Nexesenex/kobold.cpp/releases/tag/v1.55.1_b1842)
- [llamacpp_0.2.29 branch](https://github.com/oobabooga/text-generation-webui/tree/llamacpp_0.2.29) of Ooba also works
[More info about IQ2](https://github.com/ggerganov/llama.cpp/pull/4897)
# Models
Models: [IQ2-XS](https://huggingface.co/Kooten/WinterGoddess-1.4x-70B-L2-IQ2-GGUF/blob/main/WinterGoddess-1.4x-70B-L2-IQ2_XS.gguf), [IQ2-XXS](https://huggingface.co/Kooten/WinterGoddess-1.4x-70B-L2-IQ2-GGUF/blob/main/WinterGoddess-1.4x-70B-L2-IQ2_XXS.gguf)
Regular GGUF Quants: [Here](https://huggingface.co/TheBloke/WinterGoddess-1.4x-70B-L2-GGUF)
## Prompt Format
### Alpaca:
```
### Instruction:
<Prompt>
### Response:
```
OR
```
### Instruction:
<Prompt>
### Input:
<Insert Context Here>
### Response:
```
## Contact
Kooten on discord |