File size: 10,091 Bytes
a5ba918 bb0d484 a5ba918 6842f8b a5ba918 a92aec7 a5ba918 da35eef 3a74088 d9d3387 73c48ff fd47dda 30c735a a5ba918 441a0ad 34265c7 019d7bd 8335e9a d958dfa a28a072 9a5837e d958dfa 3a74088 7fcfbfd a28a072 db9ab21 44dc967 7ec12e8 db9ab21 a5ba918 bb0d484 a5ba918 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 |
---
base_model: xai-org/grok-1
language:
- en
library_name: transformers
license: apache-2.0
quantized_by: mradermacher
tags:
- grok-1
---
## About
weighted/imatrix quants of https://huggingface.co/xai-org/grok-1
<!-- provided-files -->
static quants are available at https://huggingface.co/mradermacher/grok-1-GGUF
## Usage
If you are unsure how to use GGUF files, refer to one of [TheBloke's
READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for
more details, including on how to concatenate multi-part files.
## Provided Quants
(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)
| Link | Type | Size/GB | Notes |
|:-----|:-----|--------:|:------|
| [PART 1](https://huggingface.co/mradermacher/grok-1-i1-GGUF/resolve/main/grok-1.i1-IQ1_S.gguf.part1of2) [PART 2](https://huggingface.co/mradermacher/grok-1-i1-GGUF/resolve/main/grok-1.i1-IQ1_S.gguf.part2of2) | i1-IQ1_S | 66.7 | for the desperate |
| [PART 1](https://huggingface.co/mradermacher/grok-1-i1-GGUF/resolve/main/grok-1.i1-IQ1_M.gguf.part1of2) [PART 2](https://huggingface.co/mradermacher/grok-1-i1-GGUF/resolve/main/grok-1.i1-IQ1_M.gguf.part2of2) | i1-IQ1_M | 73.7 | mostly desperate |
| [PART 1](https://huggingface.co/mradermacher/grok-1-i1-GGUF/resolve/main/grok-1.i1-IQ2_XXS.gguf.part1of2) [PART 2](https://huggingface.co/mradermacher/grok-1-i1-GGUF/resolve/main/grok-1.i1-IQ2_XXS.gguf.part2of2) | i1-IQ2_XXS | 85.4 | |
| [PART 1](https://huggingface.co/mradermacher/grok-1-i1-GGUF/resolve/main/grok-1.i1-IQ2_XS.gguf.part1of2) [PART 2](https://huggingface.co/mradermacher/grok-1-i1-GGUF/resolve/main/grok-1.i1-IQ2_XS.gguf.part2of2) | i1-IQ2_XS | 94.7 | |
| [PART 1](https://huggingface.co/mradermacher/grok-1-i1-GGUF/resolve/main/grok-1.i1-IQ2_S.gguf.part1of2) [PART 2](https://huggingface.co/mradermacher/grok-1-i1-GGUF/resolve/main/grok-1.i1-IQ2_S.gguf.part2of2) | i1-IQ2_S | 96.1 | |
| [PART 1](https://huggingface.co/mradermacher/grok-1-i1-GGUF/resolve/main/grok-1.i1-IQ2_M.gguf.part1of3) [PART 2](https://huggingface.co/mradermacher/grok-1-i1-GGUF/resolve/main/grok-1.i1-IQ2_M.gguf.part2of3) [PART 3](https://huggingface.co/mradermacher/grok-1-i1-GGUF/resolve/main/grok-1.i1-IQ2_M.gguf.part3of3) | i1-IQ2_M | 105.4 | |
| [PART 1](https://huggingface.co/mradermacher/grok-1-i1-GGUF/resolve/main/grok-1.i1-Q2_K.gguf.part1of3) [PART 2](https://huggingface.co/mradermacher/grok-1-i1-GGUF/resolve/main/grok-1.i1-Q2_K.gguf.part2of3) [PART 3](https://huggingface.co/mradermacher/grok-1-i1-GGUF/resolve/main/grok-1.i1-Q2_K.gguf.part3of3) | i1-Q2_K | 117.2 | IQ3_XXS probably better |
| [PART 1](https://huggingface.co/mradermacher/grok-1-i1-GGUF/resolve/main/grok-1.i1-IQ3_XXS.gguf.part1of3) [PART 2](https://huggingface.co/mradermacher/grok-1-i1-GGUF/resolve/main/grok-1.i1-IQ3_XXS.gguf.part2of3) [PART 3](https://huggingface.co/mradermacher/grok-1-i1-GGUF/resolve/main/grok-1.i1-IQ3_XXS.gguf.part3of3) | i1-IQ3_XXS | 123.7 | lower quality |
| [PART 1](https://huggingface.co/mradermacher/grok-1-i1-GGUF/resolve/main/grok-1.i1-IQ3_XS.gguf.part1of3) [PART 2](https://huggingface.co/mradermacher/grok-1-i1-GGUF/resolve/main/grok-1.i1-IQ3_XS.gguf.part2of3) [PART 3](https://huggingface.co/mradermacher/grok-1-i1-GGUF/resolve/main/grok-1.i1-IQ3_XS.gguf.part3of3) | i1-IQ3_XS | 130.7 | |
| [PART 1](https://huggingface.co/mradermacher/grok-1-i1-GGUF/resolve/main/grok-1.i1-IQ3_S.gguf.part1of3) [PART 2](https://huggingface.co/mradermacher/grok-1-i1-GGUF/resolve/main/grok-1.i1-IQ3_S.gguf.part2of3) [PART 3](https://huggingface.co/mradermacher/grok-1-i1-GGUF/resolve/main/grok-1.i1-IQ3_S.gguf.part3of3) | i1-IQ3_S | 138.5 | beats Q3_K* |
| [PART 1](https://huggingface.co/mradermacher/grok-1-i1-GGUF/resolve/main/grok-1.i1-Q3_K_S.gguf.part1of3) [PART 2](https://huggingface.co/mradermacher/grok-1-i1-GGUF/resolve/main/grok-1.i1-Q3_K_S.gguf.part2of3) [PART 3](https://huggingface.co/mradermacher/grok-1-i1-GGUF/resolve/main/grok-1.i1-Q3_K_S.gguf.part3of3) | i1-Q3_K_S | 138.5 | IQ3_XS probably better |
| [PART 1](https://huggingface.co/mradermacher/grok-1-i1-GGUF/resolve/main/grok-1.i1-IQ3_M.gguf.part1of4) [PART 2](https://huggingface.co/mradermacher/grok-1-i1-GGUF/resolve/main/grok-1.i1-IQ3_M.gguf.part2of4) [PART 3](https://huggingface.co/mradermacher/grok-1-i1-GGUF/resolve/main/grok-1.i1-IQ3_M.gguf.part3of4) [PART 4](https://huggingface.co/mradermacher/grok-1-i1-GGUF/resolve/main/grok-1.i1-IQ3_M.gguf.part4of4) | i1-IQ3_M | 145.3 | |
| [PART 1](https://huggingface.co/mradermacher/grok-1-i1-GGUF/resolve/main/grok-1.i1-Q3_K_M.gguf.part1of4) [PART 2](https://huggingface.co/mradermacher/grok-1-i1-GGUF/resolve/main/grok-1.i1-Q3_K_M.gguf.part2of4) [PART 3](https://huggingface.co/mradermacher/grok-1-i1-GGUF/resolve/main/grok-1.i1-Q3_K_M.gguf.part3of4) [PART 4](https://huggingface.co/mradermacher/grok-1-i1-GGUF/resolve/main/grok-1.i1-Q3_K_M.gguf.part4of4) | i1-Q3_K_M | 153.0 | IQ3_S probably better |
| [PART 1](https://huggingface.co/mradermacher/grok-1-i1-GGUF/resolve/main/grok-1.i1-Q3_K_L.gguf.part1of4) [PART 2](https://huggingface.co/mradermacher/grok-1-i1-GGUF/resolve/main/grok-1.i1-Q3_K_L.gguf.part2of4) [PART 3](https://huggingface.co/mradermacher/grok-1-i1-GGUF/resolve/main/grok-1.i1-Q3_K_L.gguf.part3of4) [PART 4](https://huggingface.co/mradermacher/grok-1-i1-GGUF/resolve/main/grok-1.i1-Q3_K_L.gguf.part4of4) | i1-Q3_K_L | 164.5 | IQ3_M probably better |
| [PART 1](https://huggingface.co/mradermacher/grok-1-i1-GGUF/resolve/main/grok-1.i1-IQ4_XS.gguf.part1of4) [PART 2](https://huggingface.co/mradermacher/grok-1-i1-GGUF/resolve/main/grok-1.i1-IQ4_XS.gguf.part2of4) [PART 3](https://huggingface.co/mradermacher/grok-1-i1-GGUF/resolve/main/grok-1.i1-IQ4_XS.gguf.part3of4) [PART 4](https://huggingface.co/mradermacher/grok-1-i1-GGUF/resolve/main/grok-1.i1-IQ4_XS.gguf.part4of4) | i1-IQ4_XS | 170.2 | |
| [PART 1](https://huggingface.co/mradermacher/grok-1-i1-GGUF/resolve/main/grok-1.i1-IQ4_NL.gguf.part1of4) [PART 2](https://huggingface.co/mradermacher/grok-1-i1-GGUF/resolve/main/grok-1.i1-IQ4_NL.gguf.part2of4) [PART 3](https://huggingface.co/mradermacher/grok-1-i1-GGUF/resolve/main/grok-1.i1-IQ4_NL.gguf.part3of4) [PART 4](https://huggingface.co/mradermacher/grok-1-i1-GGUF/resolve/main/grok-1.i1-IQ4_NL.gguf.part4of4) | i1-IQ4_NL | 180.0 | prefer IQ4_XS |
| [PART 1](https://huggingface.co/mradermacher/grok-1-i1-GGUF/resolve/main/grok-1.i1-Q4_0.gguf.part1of4) [PART 2](https://huggingface.co/mradermacher/grok-1-i1-GGUF/resolve/main/grok-1.i1-Q4_0.gguf.part2of4) [PART 3](https://huggingface.co/mradermacher/grok-1-i1-GGUF/resolve/main/grok-1.i1-Q4_0.gguf.part3of4) [PART 4](https://huggingface.co/mradermacher/grok-1-i1-GGUF/resolve/main/grok-1.i1-Q4_0.gguf.part4of4) | i1-Q4_0 | 180.5 | fast, low quality |
| [PART 1](https://huggingface.co/mradermacher/grok-1-i1-GGUF/resolve/main/grok-1.i1-Q4_K_S.gguf.part1of4) [PART 2](https://huggingface.co/mradermacher/grok-1-i1-GGUF/resolve/main/grok-1.i1-Q4_K_S.gguf.part2of4) [PART 3](https://huggingface.co/mradermacher/grok-1-i1-GGUF/resolve/main/grok-1.i1-Q4_K_S.gguf.part3of4) [PART 4](https://huggingface.co/mradermacher/grok-1-i1-GGUF/resolve/main/grok-1.i1-Q4_K_S.gguf.part4of4) | i1-Q4_K_S | 181.6 | optimal size/speed/quality |
| [PART 1](https://huggingface.co/mradermacher/grok-1-i1-GGUF/resolve/main/grok-1.i1-Q4_K_M.gguf.part1of4) [PART 2](https://huggingface.co/mradermacher/grok-1-i1-GGUF/resolve/main/grok-1.i1-Q4_K_M.gguf.part2of4) [PART 3](https://huggingface.co/mradermacher/grok-1-i1-GGUF/resolve/main/grok-1.i1-Q4_K_M.gguf.part3of4) [PART 4](https://huggingface.co/mradermacher/grok-1-i1-GGUF/resolve/main/grok-1.i1-Q4_K_M.gguf.part4of4) | i1-Q4_K_M | 193.3 | fast, recommended |
| [P1](https://huggingface.co/mradermacher/grok-1-i1-GGUF/resolve/main/grok-1.i1-Q5_K_S.gguf.part1of5) [P2](https://huggingface.co/mradermacher/grok-1-i1-GGUF/resolve/main/grok-1.i1-Q5_K_S.gguf.part2of5) [P3](https://huggingface.co/mradermacher/grok-1-i1-GGUF/resolve/main/grok-1.i1-Q5_K_S.gguf.part3of5) [P4](https://huggingface.co/mradermacher/grok-1-i1-GGUF/resolve/main/grok-1.i1-Q5_K_S.gguf.part4of5) [P5](https://huggingface.co/mradermacher/grok-1-i1-GGUF/resolve/main/grok-1.i1-Q5_K_S.gguf.part5of5) | i1-Q5_K_S | 219.1 | |
| [P1](https://huggingface.co/mradermacher/grok-1-i1-GGUF/resolve/main/grok-1.i1-Q5_K_M.gguf.part1of5) [P2](https://huggingface.co/mradermacher/grok-1-i1-GGUF/resolve/main/grok-1.i1-Q5_K_M.gguf.part2of5) [P3](https://huggingface.co/mradermacher/grok-1-i1-GGUF/resolve/main/grok-1.i1-Q5_K_M.gguf.part3of5) [P4](https://huggingface.co/mradermacher/grok-1-i1-GGUF/resolve/main/grok-1.i1-Q5_K_M.gguf.part4of5) [P5](https://huggingface.co/mradermacher/grok-1-i1-GGUF/resolve/main/grok-1.i1-Q5_K_M.gguf.part5of5) | i1-Q5_K_M | 225.9 | |
| [P1](https://huggingface.co/mradermacher/grok-1-i1-GGUF/resolve/main/grok-1.i1-Q6_K.gguf.part1of6) [P2](https://huggingface.co/mradermacher/grok-1-i1-GGUF/resolve/main/grok-1.i1-Q6_K.gguf.part2of6) [P3](https://huggingface.co/mradermacher/grok-1-i1-GGUF/resolve/main/grok-1.i1-Q6_K.gguf.part3of6) [P4](https://huggingface.co/mradermacher/grok-1-i1-GGUF/resolve/main/grok-1.i1-Q6_K.gguf.part4of6) [P5](https://huggingface.co/mradermacher/grok-1-i1-GGUF/resolve/main/grok-1.i1-Q6_K.gguf.part5of6) [P6](https://huggingface.co/mradermacher/grok-1-i1-GGUF/resolve/main/grok-1.i1-Q6_K.gguf.part6of6) | i1-Q6_K | 260.9 | practically like static Q6_K |
Here is a handy graph by ikawrakow comparing some lower-quality quant
types (lower is better):
And here are Artefact2's thoughts on the matter:
https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9
## FAQ / Model Request
See https://huggingface.co/mradermacher/model_requests for some answers to
questions you might have and/or if you want some other model quantized.
## Thanks
I thank my company, [nethype GmbH](https://www.nethype.de/), for letting
me use its servers and providing upgrades to my workstation to enable
this work in my free time.
<!-- end -->
|