metadata
license: llama2
language:
- en
Euryale-1.4-L2-70B IQ2-GGUF
Description
IQ2-GGUF quants of Sao10K/Euryale-1.4-L2-70B
Unlike regular GGUF quants this uses important matrix similar to Quip# to keep the quant from degrading too much even at 2bpw allowing you to run larger models on less powerful machines.
NOTE: Currently you will need experimental branches of Koboldcpp or Ooba for this to work.
- Nexesenex have compiled Windows binaries HERE
- llamacpp_0.2.29 branch of Ooba also works
Models
Regular GGUF Quants: Here
Prompt Format
Alpaca:
Below is an instruction that describes a task. Write a response that appropriately completes the request.
### Instruction:
{prompt}
### Input:
{input}
### Response:
Contact
Kooten on discord