metadata

datasets:
  - Oniichat/bluemoon_roleplay_chat_data_300k_messages
inference: false
language:
  - en
license: llama2
model_creator: nRuaif
model_link: https://huggingface.co/nRuaif/Mythalion-Kimiko-v2
model_name: Mythalion-Kimiko-v2
model_type: llama
pipeline_tag: text-generation
quantized_by: Eigeen
tags:
  - text generation
  - instruct
thumbnail: null

Mythalion 13B Kimiko-v2 - ExLlamaV2

Original model: Mythalion-Kimiko-v2

Description

This is my trial of quantization. I use only RP dataset for calibration, it may cause the model to not perform as well in other situations. But people who use Mythalion basically use it for RP, I guess?

Anyway, it works well on RP. I haven't tested it's performance in other situations. ExLlamaV2 is great.

6.05 bpw is designed for 16GB VRAM. If you have 24GB VRAM, you can expand the context to at least 8192. I did not calculate the exact values.