base_model: | |
- 152334H/miqu-1-70b-sf | |
language: | |
- en | |
library_name: transformers | |
quantized_by: mradermacher | |
tags: | |
- mergekit | |
- merge | |
## About | |
weighted/imatrix quants of https://huggingface.co/wolfram/miqu-1-103b | |
<!-- provided-files --> | |
## Usage | |
If you are unsure how to use GGUF files, refer to one of [TheBloke's | |
READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for | |
more details, including on how to concatenate multi-part files. | |
## Provided Quants | |
(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) | |
| Link | Type | Size/GB | Notes | | |
|:-----|:-----|--------:|:------| | |
| [GGUF](https://huggingface.co/mradermacher/miqu-1-103b-i1-GGUF/resolve/main/miqu-1-103b.i1-IQ2_M.gguf) | i1-IQ2_M | 35.0 | | | |
| [GGUF](https://huggingface.co/mradermacher/miqu-1-103b-i1-GGUF/resolve/main/miqu-1-103b.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 39.9 | fast, lower quality | | |
| [GGUF](https://huggingface.co/mradermacher/miqu-1-103b-i1-GGUF/resolve/main/miqu-1-103b.i1-IQ3_XS.gguf) | i1-IQ3_XS | 42.4 | | | |
Here is a handy graph by ikawrakow comparing some lower-quality quant | |
types (lower is better): | |
![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png) | |
<!-- end --> | |