metadata
license: apache-2.0
language:
- en
quantized_by: TheMelonGod
pipeline_tag: text-generation
tags:
- quantized
- safetensors
- exllamav2
- mistral
base_model:
- cognitivecomputations/dolphin-2.9.3-mistral-7B-32k
base_model_relation: quantized
ExLlamaV2 quantizations of: Cognitive Computations - dolphin-2.9.3-mistral-7B-32k
Quantizations (6hb)
8.0bpw
7.5bpw
7.0bpw
6.5bpw
6.0bpw
5.5bpw
5.0bpw
4.5bpw
4.25bpw
4.0bpw
3.75bpw
3.5bpw
3.0bpw
2.75bpw
2.5bpw
2.25bpw
2.0bpw
If you need a specific model quantization or a particular bits per weight, please let me know. I’m happy to help quantize lesser known models.
If you have any suggestions for improvements or feedback, feel free to reach out. Your input is greatly appreciated and helps me make quantizations better for everyone.
Special thanks to turboderp for developing the tools that made these quantizations possible. Your contributions are greatly appreciated!