Update README.md
Browse files
README.md
CHANGED
|
@@ -5,6 +5,8 @@ license: cc-by-nc-4.0
|
|
| 5 |
# Noromaid-v0.4-Mixtral-Instruct-8x7b-Zloss 3.5bpw
|
| 6 |
Exllama quant of [NeverSleep/Noromaid-v0.4-Mixtral-Instruct-8x7b-Zloss](https://huggingface.co/NeverSleep/Noromaid-v0.4-Mixtral-Instruct-8x7b-Zloss)
|
| 7 |
|
|
|
|
|
|
|
| 8 |
|
| 9 |
### Prompt format: Chatml
|
| 10 |
```
|
|
|
|
| 5 |
# Noromaid-v0.4-Mixtral-Instruct-8x7b-Zloss 3.5bpw
|
| 6 |
Exllama quant of [NeverSleep/Noromaid-v0.4-Mixtral-Instruct-8x7b-Zloss](https://huggingface.co/NeverSleep/Noromaid-v0.4-Mixtral-Instruct-8x7b-Zloss)
|
| 7 |
|
| 8 |
+
You will need 24gb of vram to run this model at about half context (16k, you can probably go a bit higher too)
|
| 9 |
+
|
| 10 |
|
| 11 |
### Prompt format: Chatml
|
| 12 |
```
|