Mythalion 13B Kimiko-v2 - ExLlamaV2
Original model: Mythalion-Kimiko-v2
Description
This is my trial of quantization. I use only RP dataset for calibration, it may cause the model to not perform as well in other situations. But people who use Mythalion basically use it for RP, I guess?
Anyway, it works well on RP. I haven't tested it's performance in other situations. ExLlamaV2 is great.
6.05 bpw is designed for 16GB VRAM. If you have 24GB VRAM, you can expand the context to at least 8192. I did not calculate the exact values.
- Downloads last month
- 9