README.md

---
license: cc-by-nc-4.0
tags:
- not-for-all-audiences
- nsfw
---
## Description

Exllama 2 quant of [Undi95/Lewd-Sydney-20B](https://huggingface.co/Undi95/Lewd-Sydney-20B)

4 BPW, Head bit set to 8


## VRAM
My VRAM usage with 20B models are:
| Bits per weight  | Context | VRAM  |
|--|--|--|
| 6bpw | 4k | 24gb | 
| 4bpw | 4k | 18gb | 
| 4bpw | 8k | 24gb | 
| 3bpw | 4k | 16gb | 
| 3bpw | 8k | 21gb | 
I have rounded up, these arent exact numbers, this is also on a windows machine.

## Prompt template: Alpaca

```
Below is an instruction that describes a task. Write a response that appropriately completes the request.

### Instruction:
{prompt}

### Response:

```