Description
Exllama 2 quant of tavtav/Rose-20B
Please make sure to read their description
3 BPW, Head bit set to 8
After having tried different values for the hb i found it makes little difference in vram usage (20-50mb more from 6 to 8) so i will keep leaving it at 8
Prompt template: Custom format, or Alpaca
Custom format:
Since this is based on Noromaid i recommend the SillyTavern config files from NeverSleep:
Alpaca:
Below is an instruction that describes a task. Write a response that appropriately completes the request.
### Instruction:
{prompt}
### Response:
- Downloads last month
- 21
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social
visibility and check back later, or deploy to Inference Endpoints (dedicated)
instead.